Realtime Data Distribution via SOAP?

Hi all,

Has anyone used SOAP to create a high frequency asynchronous real-time data program which will be scalable for multiple clients?

I'm looking to create a program to send realtime trade signals and my concern with SOAP is whether it will be able to cope with high frequency asynchronous updates (in this case I will be looking at STIRs and Bund/Bobl/Schatz contracts)

RichieE said:

Hi all,

Has anyone used SOAP to create a high frequency asynchronous real-time data program which will be scalable for multiple clients?

I'm looking to create a program to send realtime trade signals and my concern with SOAP is whether it will be able to cope with high frequency asynchronous updates (in this case I will be looking at STIRs and Bund/Bobl/Schatz contracts)

Click to expand...

What is your definition of high frequency? once per day? once per hour? 100 times a second?

You should be good for around 5 times a second, probably more depending on how big the paylaod is
How many people are you sending out to?

Robertral said:

What is your definition of high frequency? once per day? once per hour? 100 times a second?

You should be good for around 5 times a second, probably more depending on how big the paylaod is
How many people are you sending out to?

Click to expand...

Hi Robertal, I'm looking at a max of about 5 times a second per ticker in a busy market. This is a live trading app with a max of approx. 10 'listeners' (clients) monitoring the signals coming from the server application. I've been investigating other ways to achieve the same goal VB.NET suggests that I should be able to use the SOCKETS method too, but I'm not familiar with sockets programming at all.

RichieE said:

Hi Robertal, I'm looking at a max of about 5 times a second per ticker in a busy market. This is a live trading app with a max of approx. 10 'listeners' (clients) monitoring the signals coming from the server application. I've been investigating other ways to achieve the same goal VB.NET suggests that I should be able to use the SOCKETS method too, but I'm not familiar with sockets programming at all.

Click to expand...

If you are not familiar with sockets you really need to learn some more about network programming. It is basic and required knowledge. Stevens book on Unix network programming is recommended - even if you are working on windows.

I would suggest that SOAP is not the tool for this job. XML is too "heavy" for this sort of thing. In any case SOAP is mostly an RPC type mechanism, which is not really what you are trying to do.

If you want to do this right you might have a look at FIX and the related FAST. These are where big players are going.

If you want to see how others do this sort of thing, you could have a look at opentick :: home The protocol they use is open and the specifications are published on the web site.

dcraig1 said:

If you are not familiar with sockets you really need to learn some more about network programming. It is basic and required knowledge. Stevens book on Unix network programming is recommended - even if you are working on windows.

Click to expand...

Thanks for that, I'll be off to the bookshop today!

dcraig1 said:

I would suggest that SOAP is not the tool for this job. XML is too "heavy" for this sort of thing. In any case SOAP is mostly an RPC type mechanism, which is not really what you are trying to do.

Click to expand...

Great, I've been looking for some indication as to whether SOAP will be lean enough to run a realtime application. It has been surprisingly difficult to find out.

dcraig1 said:

If you want to do this right you might have a look at FIX and the related FAST. These are where big players are going.

Click to expand...

I will investigate

dcraig1 said:

If you want to see how others do this sort of thing, you could have a look at opentick :: home The protocol they use is open and the specifications are published on the web site.

Click to expand...

I've browsed through the Opentick website but they don't seem to have any open source code for the server side... only the client side. (Unless I've missed something that is...)

Hi RichieE,

What are your trade signals in STIRS based on? Interested to see what you come up with.

I've browsed through the Opentick website but they don't seem to have any open source code for the server side... only the client side. (Unless I've missed something that is...)

Click to expand...

Yes, that's so, but the protocol spec is there somewhere. I haven't read it, but it should give you an idea as to how the messages are assembled and a kick start in designing your own protocol, should you choose to go down that path.

Of course there are many ways of going about this sort of thing, but I would be cautious of "hand assembling" messages. ie bytes 1 - 4 is the bid size etc etc. If you are able to use something that automatically encodes/decodes (or marshalls) the wire protocol is does make life easier.

For simple elegance and high performance, I always liked the XDR protocol as used in SUN's RPC. It is an internet standard (there is an RFC), and the compiler "rpcgen" is available on just about all *nix systems. Unfortunately it is oriented towards C, so is probably not want you want, but is a good example of what I am talking about.

Lucky_81 said:

Hi RichieE,

What are your trade signals in STIRS based on? Interested to see what you come up with.

Click to expand...

Hi Lucky, we're still working on that part! Backtesting is in progress... I'm working as part of a group of prop traders and the original trade ideas are not mine; therefore I don't feel that it would be right to reveal someone else's work on a public forum. However, I suspect the ideas are not entirely original so a little research will probably uncover very similar concepts.

dcraig1 said:

Yes, that's so, but the protocol spec is there somewhere. I haven't read it, but it should give you an idea as to how the messages are assembled and a kick start in designing your own protocol, should you choose to go down that path.

Click to expand...

That was very interesting reading. I'm really grateful for that hint.

dcraig1 said:

Of course there are many ways of going about this sort of thing, but I would be cautious of "hand assembling" messages. ie bytes 1 - 4 is the bid size etc etc.

Click to expand...

Why do you say that exactly? Is this solely to remove the admin of keeping a proprietary protocol? Or are there other issues?

dcraig1 said:

If you are able to use something that automatically encodes/decodes (or marshalls) the wire protocol is does make life easier.

Click to expand...

That would be perfect. Right now my dream scenario would be to find an ActiveX Control or class on the internet that I could incorporate into my software. This would act as a "tunnel" between my 'server' and my 'client' PCs. I simply want to call a method on the server passing a price data and a timestamp. This will then raise an event in the client PCs which will process that price and timestamp.

dcraig1 said:

For simple elegance and high performance, I always liked the XDR protocol as used in SUN's RPC. It is an internet standard (there is an RFC), and the compiler "rpcgen" is available on just about all *nix systems. Unfortunately it is oriented towards C, so is probably not want you want, but is a good example of what I am talking about.

Click to expand...

I've just taken a look at this and it does seem to delve into the fine mechanics... surely there must be a premade 'tunnel' plugin code that I could 'nab'? Do you think I will be wasting my time searching for such a tool?