Wednesday 23 August 2017

PostgreSQL Londiste3 Replication

I started to experiment with Skytools3, which is the database management tool from Skype. The replication tool is called Londiste3. I followed the documentation to setup & start testing with it. However, I found the documentation not 100% self-explanatory. By going through few try and google around, I manage to install and make it work.
Note ==> Here Blue color syntax optional
(A) Python AND Skytools3 Installation  ( Need to check current available rpm)
python-psycopg2-2.0.14-1.rhel5.x86_64.rpm
skytools-*.rpm
skytools-modules-*.x86_64.rpm

(B) Replication Setup
These steps are for simple replication from 1 master to 1 slave. According to Londiste term, each master database is called "root node", slave is "leaf node". You can view the original documentation here.

Assuming the below scenario:
master (root) server IP: 192.168.90.22
Slave (leaf) server IP: 192.168.90.23
database: testdb
table: table_test

1) On root (192.168.90.22)node, create Londiste's config file & root node, then run worker process for root node:

create config file - root node
·       vi /var/lib/pgsql/primary.ini   
[londiste3]
job_name = primary
db = dbname=testdb host=192.168.90.22 port=5432 user=postgres password=1234
queue_name = replication
logfile = /var/lib/pgsql/londiste_log/%(job_name)s.log
pidfile = /var/lib/pgsql/londiste_run/%(job_name)s.pid

create root node
londiste3 /var/lib/pgsql/primary.ini create-root primary "dbname=testdb host=192.168.90.22 port=5432 user=postgres password=1234"

run worker process
londiste3 -d /var/lib/pgsql/primary.ini worker

check worker process running
Worker process will now run in background as daemon. You can check with this:

pgrep -lf londiste
13795 /usr/bin/python /usr/bin/londiste3 -d /var/lib/pgsql/primary.ini worker

2) On leaf (192.168.90.23) node, create Londiste's config file & leaf node, then run worker process for leaf node:

create config file - leaf node
·       vi /var/lib/pgsql/subscriber.ini
[londiste3]
job_name = subscriber
db = dbname=testdb host=192.168.90.23 port=5432 user=postgres password=1234
queue_name = replication
logfile = /var/lib/pgsql/londiste_log/%(job_name)s.log
pidfile = /var/lib/pgsql/londiste_run/%(job_name)s.pid

create leaf node
londiste3 /var/lib/pgsql/subscriber.ini create-leaf subscriber "dbname=testdb host=192.168.90.23" --provider="dbname=testdb host=192.168.90.22 port=5432"

run worker daemon for leaf node
londiste3 -d /var/lib/pgsql/subscriber.ini worker

3) Setup PgQ Ticker Daemon on root node

Create ticker daemon config file
·       vi /var/lib/pgsql/pgqd.ini
[pgqd]
logfile = /var/lib/pgsql/londiste_log/pgqd.log
pidfile = /var/lib/pgsql/londiste_run/pgqd.pid
base_connstr = dbname=testdb host=192.168.90.22 port=5432 user=postgres password=1234

Start ticker daemon
pgqd -d /var/lib/pgsql/pgqd.ini

4) Now we have to tell Londiste which table to replicate:

On root node
londiste3 /var/lib/pgsql/primary.ini add-table table_test

On leaf node
londiste3 /var/lib/pgsql/subscriber.ini add-table table_test

Replication setup is now done! You can insert some rows into table on master db, & see them reach it's slave :)

*****************************************************************************
Londiste2
=========

Londiste3
=========

No comments:

Post a Comment