配置Goldengate从Oracle到PostgreSQL的同步复制

1、平台环境

Oracle：rhel6.7+Oracle11.2.0.4 ip:192.168.56.2

PostgreSQL：rhel7.2+Pg9.6.1 ip:192.168.56.25

Goldengate：Goldengate12.2.0.1 for oracle和Goldengate 12.2.0.1 for PostgreSQL

2、Ogg配置

Oracle端：

直接安装goldengate for oracle 11g

配置环境变量

[oracle@rhel6ogg]$vi~/.bash_profile
#添加
exportLD_LIBRARY_PATH=/ogg/lib:$LD_LIBRARY_PATH
exportPATH=/ogg:$PATH

配置Oracle数据库

#启用归档
sys@ORCL>alterdatabasearchivelog;
#Forcinglogging
sys@ORCL>alterdatabaseforcelogging;
#添加最小附加日志
sys@ORCL>alterdatabaseaddsupplementallogdata;
#查看结果
sys@ORCL>selectLOG_MODE,FORCE_LOGGING,SUPPLEMENTAL_LOG_DATA_MINfromv$database;

LOG_MODE			FORCE_LOGSUPPLEMENTAL_LOG_DATA_MI
---------------------------------------------------------------------
ARCHIVELOG			YESYES
#创建goldengate用户
sys@ORCL>createusergoldengateidentifiedbygoldengate;
sys@ORCL>grantdbatogoldengate;
sys@ORCL>createuserzhaoxuidentifiedbyzhaoxu;
sys@ORCL>grantdbatozhaoxu;
#创建测试表
zhaoxu@ORCL>createtableggtest(col1number,col2varchar2(20));

Tablecreated.

zhaoxu@ORCL>altertableggtestaddconstraintpk_ggtestprimarykey(col1);

Tablealtered.

配置ogg参数文件

#配置mgr
GGSCI(rhel6)2>editparamsmgr
PORT7809
AUTOSTARTER*
AUTORESTARTEXTRACT*,RETRIES100,WAITMINUTES2
PURGEOLDEXTRACTS./dirdat/*,USECHECKPOINTS,MINKEEPDAYS3
LAGREPORTHOURS1
LAGINFOMINUTES30
LAGCRITICALMINUTES45
SYSLOGERROR,WARN

#启动mgr
GGSCI(rhel6)3>startmgr
GGSCI(rhel6)4>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING

#配置抽取进程参数
GGSCI(rhel6)5>editparamsext_emp
EXTRACTEXT_EMP
DYNAMICRESOLUTION
SETENV(NLS_LANG="AMERICAN_AMERICA.UTF8")
SETENV(ORACLE_HOME="/u02/app/oracle/product/11.2.4/db1")
SETENV(ORACLE_SID="orcl")
USERIDgoldengate,PASSWORDgoldengate
DISCARDFILE./dirrpt/ext_emp.dsc,APPEND,MEGABYTES1024
EXTTRAIL./dirdat/zx
tablezhaoxu.ggtest;

#配置投递进程参数
GGSCI(rhel6)6>editparamsdp_tab
EXTRACTDP_TAB
PASSTHRU
RMTHOST192.168.56.25,MGRPORT7809,COMPRESS
RMTTRAIL./dirdat/zx
tablezhaoxu.ggtest;

#配置生成定义文件参数
GGSCI(rhel6)7>editparamsdefgen
defsfile./dirdef/defgen.def
useridgoldengate,passwordgoldengate
tablezhaoxu.ggtest;

#增加抽取进程和传输进程
GGSCI(rhel6)8>addextractext_emp,tranlog,beginnow
GGSCI(rhel6)9>addexttrail./dirdat/zx,extractext_emp,megabytes200
GGSCI(rhel6)10>addextractdp_tab,exttrailsource./dirdat/zx
GGSCI(rhel6)11>addrmttrail./dirdat/zx,extractdp_tab,megabytes200
GGSCI(rhel6)12>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING
EXTRACTSTOPPEDDP_TAB00:00:0000:01:43
EXTRACTSTOPPEDEXT_EMP00:00:0000:01:01

#添加表的附加日志
GGSCI(rhel6)13>dbloginuseridgoldengatepasswordgoldengate
Successfullyloggedintodatabase.

GGSCI(rhel6asgoldengate@orcl)14>addtrandatazhaoxu.ggtest

LoggingofsupplementalredodataenabledfortableZHAOXU.GGTEST.
TRANDATAforschedulingcolumnshasbeenaddedontable'ZHAOXU.GGTEST'.
TRANDATAforinstantiationCSNhasbeenaddedontable'ZHAOXU.GGTEST'.

#生成定义文件
[oracle@rhel6ogg]$./defgenparamfile./dirprm/defgen.prm

***********************************************************************
OracleGoldenGateTableDefinitionGeneratorforOracle
Version12.2.0.1.1OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401
Linux,x64,64bit(optimized),Oracle11gonDec11201521:37:21

Copyright(C)1995,2015,Oracleand/oritsaffiliates.Allrightsreserved.


Startingat2016-12-0813:45:00
***********************************************************************

OperatingSystemVersion:
Linux
Version#1SMPWedJul118:23:37EDT2015,Release2.6.32-573.el6.x86_64
Node:rhel6
Machine:x86_64
softlimithardlimit
AddressSpaceSize:unlimitedunlimited
HeapSize:unlimitedunlimited
FileSize:unlimitedunlimited
CPUTime:unlimitedunlimited

Processid:3669

***********************************************************************
**Runningwiththefollowingparameters**
***********************************************************************
defsfile./dirdef/defgen.def
useridgoldengate,password***
tablezhaoxu.ggtest;
RetrievingdefinitionforZHAOXU.GGTEST.


Definitionsgeneratedfor1tablein./dirdef/defgen.def.

[oracle@rhel6ogg]$cat./dirdef/defgen.def
*+-Defgenversion5.0,EncodingUTF-8
*
*Definitionscreated/modified2016-12-0813:45
*
*Fielddescriptionsforeachcolumnentry:
*
*1Name
*2DataType
*3ExternalLength
*4FetchOffset
*5Scale
*6Level
*7Null
*8BumpifOdd
*9InternalLength
*10BinaryLength
*11TableLength
*12MostSignificantDT
*13LeastSignificantDT
*14HighPrecision
*15LowPrecision
*16ElementaryItem
*17Occurs
*18KeyColumn
*19SubDataType
*20NativeDataType
*21CharacterSet
*22CharacterLength
*23LOBType
*24PartialType
*
Databasetype:ORACLE
CharactersetID:UTF-8
NationalcharactersetID:UTF-16
Locale:neutral
Casesensitivity:14141414141414141414141411141414
TimeZone:GMT
*
DefinitionfortableZHAOXU.GGTEST
Recordlength:82
Syskey:0
Columns:2
COL1645000010505050000010122-1000
COL2642056001020200000010001-1000
Endofdefinition

PostgreSQL端：

创建用于同步的数据库、用户和Schema，并创建测试表

postgres=#createdatabasezhaoxu;
postgres=#createuserzhaoxusuperuserpassword'zhaoxu';
postgres=#\czhaoxuzhaoxu
zhaoxu=#createschemazhaoxu;
CREATESCHEMA
zhaoxu=#\dn
Listofschemas
Name|Owner
--------+--------
public|pguser
zhaoxu|zhaoxu
zhaoxu=#CREATETABLEggtest
zhaoxu-#(
zhaoxu(#col1integerNOTNULL,zhaoxu(#col2varchar(20),zhaoxu(#CONSTRAINTpk_ggtestPRIMARYKEY(col1)
zhaoxu(#);
CREATETABLE
zhaoxu=#\d
Listofrelations
Schema|Name|Type|Owner
--------+--------+-------+--------
zhaoxu|ggtest|table|zhaoxu

解压ogg

[pguser@rhel7ogg]$tar-xvfggs_Linux_x64_PostgreSQL_64bit.tar

配置odbc数据源，goldengate 使用ODBC连接Postgres Database

[pguser@rhel7ogg]$pwd
/ogg
[pguser@rhel7ogg]$catodbc.ini
[ODBCDataSources]
GG_Postgres=DataDirect9.6PostgreSQLWireProtocol
[ODBC]
IANAAppCodePage=106
InstallDir=/ogg
[GG_Postgres]
Driver=/ogg/lib/GGpsql25.so
Description=DataDirect9.6PostgreSQLWireProtocol
Database=zhaoxu
HostName=127.0.0.1
PortNumber=5432
LogonID=zhaoxu
Password=zhaoxu

[ODBCDataSources]里边配置该ODBC的别名，本文件中也就是GG_Postgres 后边的配置文件中的targetdb需要与这个对应

[ODBC]:

IANAAppCodePage指的是字符集的设置这里的106值得是UTF8，如果是4则为ISO-8859-1，注意这个应该始终和postgres的字符集设置相同，不同字符集对应的值见附件。

InstallDir对应ogg的安装目录

[GG_Postgres]:这里的名称对应的是上边ODBC的别名

Driver这里指向的是ogg安装目录下的lib/GGpsql25.so

Description是描述

Database填写数据库名称

HostName填写本机的hostname，可以解析的即可。

PosrNumber是postgres的监听端口。

LogonID填写postgres的用户名

password填写postgres的密码

配置环境变量

exportLD_LIBRARY_PATH=/ogg/lib:$LD_LIBRARY_PATH
exportPATH=$PATH:/ogg
exportODBCINI=/ogg/odbc.ini

配置Ogg

[pguser@rhel7ogg]$./ggsci

OracleGoldenGateCommandInterpreter
Version12.2.0.1.1OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401
Linux,PostgreSQLonDec11201516:22:42
OperatingsystemcharactersetidentifiedasUTF-8.

Copyright(C)1995,Oracleand/oritsaffiliates.Allrightsreserved.


#创建目录
GGSCI(rhel7)1>createsubdirs

Creatingsubdirectoriesundercurrentdirectory/ogg

Parameterfiles/ogg/dirprm:alreadyexists
Reportfiles/ogg/dirrpt:created
Checkpointfiles/ogg/dirchk:created
Processstatusfiles/ogg/dirpcs:created
SQLscriptfiles/ogg/dirsql:created
Databasedefinitionsfiles/ogg/dirdef:created
Extractdatafiles/ogg/dirdat:created
Temporaryfiles/ogg/dirtmp:created
Stdoutfiles/ogg/dirout:created

#配置mgr进程
PORT7809

#启动mgr进程
GGSCI(rhel7)3>startmgr
Managerstarted.


GGSCI(rhel7)4>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING

#把源端生成的定义文件取到目标端
[pguser@rhel7ogg]$scporacle@192.168.56.2:/ogg/dirdef/defgen.def/ogg/dirdef

#配置复制进程参数
GGSCI(rhel7)5>editparamsrep1
REPLICATrep1
SOURCEDEFS./dirdef/defgen.def
SETENV(PGCLIENTENCODING="UTF8")
SETENV(ODBCINI="/ogg/odbc.ini")
SETENV(NLS_LANG="AMERICAN_AMERICA.AL32UTF8")
TARGETDBGG_Postgres,useridzhaoxu,passwordzhaoxu
DISCARDFILE./dirrpt/rep1.dsc
mapzhaoxu.ggtest,targetzhaoxu.ggtest;

#添加复制进程
GGSCI(rhel7)6>addreplicatrep1,exttrail./dirdat/zx,nodbcheckpoint

GGSCI(rhel7)7>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING
REPLICATSTOPPEDREP100:00:0000:02:29

#测试连接PostgreSQL数据库
GGSCI(rhel7)8>dbloginsourcedbgg_postgresuseridzhaoxu
Password:

2016-12-0813:27:34INFOOGG-03036DatabasecharactersetidentifiedasUTF-8.Locale:en_US.

2016-12-0813:27:34INFOOGG-03037SessioncharactersetidentifiedasUTF-8.
Successfullyloggedintodatabase.

GGSCI(rhel7aszhaoxu@gg_postgres)9>
#如果连接不成功，检查pg_hba.conf配置文件

3、启动源端和目标端的进程

#Oracle端
GGSCI(rhel6)16>start*

SendingSTARTrequesttoMANAGER...
EXTRACTDP_TABstarting

SendingSTARTrequesttoMANAGER...
EXTRACTEXT_EMPstarting

GGSCI(rhel6)18>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING
EXTRACTRUNNINGDP_TAB00:00:0000:00:12
EXTRACTRUNNINGEXT_EMP00:00:0000:00:01

#PostgreSQL端
GGSCI(rhel7)8>start*

SendingSTARTrequesttoMANAGER...
REPLICATREP1starting


GGSCI(rhel7)9>infoall

ProgramStatusGroupLagatChkptTimeSinceChkpt

MANAGERRUNNING
REPLICATRUNNINGREP100:00:0000:00:00

4、测试数据同步

测试insert

#Oracle端
zhaoxu@ORCL>insertintoggtestvalues(1,'zhaoxu');

1rowcreated.

zhaoxu@ORCL>insertintoggtestvalues(2,'luoxi');

1rowcreated.

zhaoxu@ORCL>insertintoggtestvalues(3,'sanqi');

1rowcreated.

zhaoxu@ORCL>commit;

Commitcomplete.

#PostgreSQL端
zhaoxu=#select*fromggtest;
col1|col2
------+--------
1|zhaoxu
2|luoxi
3|sanqi
(3rows)

测试delete

#Oracle端
zhaoxu@ORCL>deletefromggtestwherecol1=3;

1rowdeleted.

zhaoxu@ORCL>commit;

Commitcomplete.

zhaoxu@ORCL>select*fromggtest;

COL1COL2
----------------------------------------------------------------------
	1zhaoxu
	2luoxi
#PostgreSQL端
zhaoxu=#select*fromggtest;
col1|col2
------+--------
1|zhaoxu
2|luoxi
(2rows)

测试update

#Oracle端
zhaoxu@ORCL>updateggtestsetcol2='sanqi'wherecol1=1;

1rowupdated.

zhaoxu@ORCL>commit;

Commitcomplete.

zhaoxu@ORCL>select*fromggtest;

COL1COL2
----------------------------------------------------------------------
	1sanqi
	2luoxi
#PostgreSQL端
zhaoxu=#select*fromggtest;
col1|col2
------+-------
2|luoxi
1|sanqi
(2rows)

参考文档：

http://www.jb51.cc/article/p-rmxbqfiq-ot.html

http://www.jb51.cc/article/p-yoikauiw-gp.html

官方文档：

http://docs.oracle.com/goldengate/c1221/gg-winux/GIPSQ/sysreq.htm#GIPSQ107

配置Goldengate从Oracle到PostgreSQL的同步复制

相关推荐