博客
关于我
pg_bulkload
阅读量:388 次
发布时间:2019-03-05

本文共 2133 字,大约阅读时间需要 7 分钟。

作者:瀚高PG实验室 (Highgo PG Lab)- 天蝎座

[highgo@hgdb ~]$ pg_config |grep CONFIGCONFIGURE = '--prefix=/opt/HighGo/db/hgdb4_enterprise' '--enable-nls=zh_CN zh_TW' '--with-perl' '--with-python' '--without-tcl' '--without-gssapi' '--without-pam' '--with-ldap' '--without-bonjour' '--with-openssl' '--with-libxml' '--with-libxslt' '--enable-thread-safety' '--with-zlib' '--without-selinux' '--with-ossp-uuid' '--with-pgport=5866' '--with-hgdb-extra-version=Enterprise Edition' 'LDFLAGS=-llber'highgo=# \d myt             Table "public.myt" Column |         Type          | Modifiers --------+-----------------------+----------- id     | integer               |  nm     | character varying(20) | highgo=# \! head /tmp/file.csv1,Fa,Shighgo=# truncate myt;TRUNCATE TABLEhighgo=# copy myt from '/tmp/file.csv' with csv;ERROR:  22P02: invalid input syntax for integer: "a"CONTEXT:  COPY myt, line 2, column id: "a"highgo=# select * from myt; id | nm ----+----(0 rows)

报错以后发现,第二行出错了,之前的行也看不到数据。

如果在大量的数据copy时,假如出现问题,那么整个copy过程都是失败的,
并且数据实际占用磁盘空间,但是无法访问。
如果我们想继续加载过程而忽略错误,可以使用pg_bulkload工具。

highgo=# create extension pg_bulkload;CREATE EXTENSIONhighgo=# select * from pg_available_extensions where name='pg_bulkload';    name     | default_version | installed_version |                             comment                             -------------+-----------------+-------------------+------------------------------------------------ pg_bulkload | 1.0             |                   | pg_bulkload is a high speed data loading utility for PostgreSQL(1 row)highgo=# create extension pg_bulkload;CREATE EXTENSION[highgo@hgdb ~]$ cat /tmp/file.ctl TYPE = CSVINPUT = /tmp/file.csv			  DELIMITER=","                     #分隔符TABLE = myt         LOGFILE = /tmp/blk.log			  #日志文件PARSE_BADFILE = /tmp/parse.csv    #坏数据记录[highgo@hgdb ~]$ pg_bulkload /tmp/file.ctl -d highgoNOTICE: BULK LOAD STARTNOTICE: BULK LOAD END	0 Rows skipped.	1 Rows successfully loaded.	1 Rows not loaded due to parse errors.	0 Rows not loaded due to duplicate errors.	0 Rows replaced with new rows.WARNING: some rows were not loaded due to errors.[highgo@hgdb ~]$ more /tmp/parse.csva,S

转载地址:http://oiowz.baihongyu.com/

你可能感兴趣的文章
multivariate_normal TypeError: ufunc ‘add‘ output (typecode ‘O‘) could not be coerced to provided……
查看>>
MySQL DBA 数据库优化策略
查看>>
multi_index_container
查看>>
mutiplemap 总结
查看>>
MySQL Error Handling in Stored Procedures---转载
查看>>
MVC 区域功能
查看>>
MySQL FEDERATED 提示
查看>>
mysql generic安装_MySQL 5.6 Generic Binary安装与配置_MySQL
查看>>
Mysql group by
查看>>
MySQL I 有福啦,窗口函数大大提高了取数的效率!
查看>>
mysql id自动增长 初始值 Mysql重置auto_increment初始值
查看>>
MySQL in 太多过慢的 3 种解决方案
查看>>
Mysql Innodb 锁机制
查看>>
MySQL InnoDB中意向锁的作用及原理探
查看>>
MySQL InnoDB事务隔离级别与锁机制深入解析
查看>>
Mysql InnoDB存储引擎 —— 数据页
查看>>
Mysql InnoDB存储引擎中的checkpoint技术
查看>>
Mysql InnoDB存储引擎中缓冲池Buffer Pool、Redo Log、Bin Log、Undo Log、Channge Buffer
查看>>
MySQL InnoDB引擎的锁机制详解
查看>>
Mysql INNODB引擎行锁的3种算法 Record Lock Next-Key Lock Grap Lock
查看>>