Does anyone know how what the best way to synchronize OpenInsight database over the internet?

I am currently looking into possible ways of synchronizing large OpenInsight database between two servers over the internet. We need this to increase the reliability i.e. if the primary server goes down for some reason secondary would automatically take over the role of the primary server.

The most obvious way to me is to have delta changes sent every time the database is modified. I am not sure how practical this approach is, since OpenInsight databases seem to change dramatically with every time it is modified. This approach may be is least cost effective.

Another approach would probably be trapping calls such as write and delete at MFS level and then creating some sort of the batch file that would reflect all modification operations in it. This file is then read by the secondary server that would execute the file and bring its database up to speed.

Does anyone have better ideas or more details on how this could be realized? Any help would be greatly appreciated.

Thank You,

Boris Kravtsov

It depends on how much work you actually want to put in. For example you could have an MFS that triggers an instantaneous SOAP update of the remote database. We guess the major question here is "Will both systems be used or should the secondary system be updated only from the primary system". In this event this idea would work well. The batched idea would also work but would introduce an obvious delay. Another approach might be to have the MFS write consecutively to the real file and a transfer file and have a process running that scanned the transfer file and used HTTP to update the remote location automatically. This would ensure minimum disruption to the local write whilst ensuring the remote system was updated expeditiously.

Either way it sounds like a fun project.

Ahhh just noticed you're using 16 bit OI. OK, you'll need a WDP then but the main thrust still applies.

The Sprezzatura Group

World Leaders in all things RevSoft

Thank you for your quick reply.

Could I ask you to elaborate more on the XML approach?

What we would ideally want is to have two servers running in Australia and US servicing those areas as well as keep the database on both synchronized. So if the failure should occur all HTTP requests (SWEB) are forwarded to the alternative server.

Is it possible to have two processes accessing one database one of which would service SOAP updates from the remote location and another would perform system?s normal tasks? What would be the architectural view of such system?

Is it really practical synchronizing constantly changing 160Mb OpenInsight database over the HTTP?

Thanks again,

Boris Kravtsov

Boris

We'll have to keep this relatively generic as this maps fairly closely onto some existing consultancy projects we are currently involved with and it would not be seemly for us to provide gratis what we are charging another organisation for.

What we would ideally want is to have two servers running in Australia and US servicing those areas as well as keep the database on both synchronized.

Not in and of itself a problem.

So if the failure should occur all HTTP requests (SWEB) are forwarded to the alternative server.

If your reference to S/Web is intentional then the job will be easier as the S/Web engine is capable of relatively easy modification to deal with these sort of issues. Specialist versions have been implemented for other clients which involve nested firewalls and n-bit encryption so a simple redirection on failure would be eminently achievable.

Is it possible to have two processes accessing one database one of which would service SOAP updates from the remote location and another would perform systems normal tasks?

The S/Web engine is already multi-tasking - one engine may be tidying up temporary files whilst another services a report request and another generates new pages. Internally we are developing/have developed complete HTTP Services APIs, XML and DOM APIs for OpenInsight which make these sort of developments easier to manage.

What would be the architectural view of such system?

This would depend on the starting point, the budget and some fairly extensive meetings.

Could you confirm that you are using OI 16 bit?

Is it really practical synchronizing constantly changing 160Mb OpenInsight database over the HTTP?.

The joy of Linear Hash is that only small amounts of data actually change - it's isn't like some DBs where a frame of half a meg has to be transmitted for every change. However, the answer to this ultimately is going to depend upon the bandwidth between the two sites, the network latency and the change volumes.

The Sprezzatura Group

World Leaders in all things RevSoft

We are going to migrate to OI 32 fairly soon, before the synchronization project is going to start. So no, we are planning to run it on OI 32

Thank you,

Boris Kravtsov

OK this makes the SOAP stuff easier is all

At the UK RUG yesterday we discussed exactly this requirement and came down to an approach which pretty much replicates what we suggested earlier!

The Sprezzatura Group

World Leaders in all things RevSoft

World Leaders in all things RevSoft

View this thread on the forum...