Realtime Data Distribution via SOAP?

[RichieE]

Hi all,

Has anyone used SOAP to create a high frequency asynchronous real-time data program which will be scalable for multiple clients?

I'm looking to create a program to send realtime trade signals and my concern with SOAP is whether it will be able to cope with high frequency asynchronous updates (in this case I will be looking at STIRs and Bund/Bobl/Schatz contracts)

 

[Robertral]

Hi all,

Has anyone used SOAP to create a high frequency asynchronous real-time data program which will be scalable for multiple clients?

I'm looking to create a program to send realtime trade signals and my concern with SOAP is whether it will be able to cope with high frequency asynchronous updates (in this case I will be looking at STIRs and Bund/Bobl/Schatz contracts)

What is your definition of high frequency? once per day? once per hour? 100 times a second?

You should be good for around 5 times a second, probably more depending on how big the paylaod is
How many people are you sending out to?

 

[RichieE]

What is your definition of high frequency? once per day? once per hour? 100 times a second?

You should be good for around 5 times a second, probably more depending on how big the paylaod is
How many people are you sending out to?

Hi Robertal, I'm looking at a max of about 5 times a second per ticker in a busy market. This is a live trading app with a max of approx. 10 'listeners' (clients) monitoring the signals coming from the server application. I've been investigating other ways to achieve the same goal VB.NET suggests that I should be able to use the SOCKETS method too, but I'm not familiar with sockets programming at all.

 

[dcraig1]

Hi Robertal, I'm looking at a max of about 5 times a second per ticker in a busy market. This is a live trading app with a max of approx. 10 'listeners' (clients) monitoring the signals coming from the server application. I've been investigating other ways to achieve the same goal VB.NET suggests that I should be able to use the SOCKETS method too, but I'm not familiar with sockets programming at all.

If you are not familiar with sockets you really need to learn some more about network programming. It is basic and required knowledge. Stevens book on Unix network programming is recommended - even if you are working on windows.

I would suggest that SOAP is not the tool for this job. XML is too "heavy" for this sort of thing. In any case SOAP is mostly an RPC type mechanism, which is not really what you are trying to do.

If you want to do this right you might have a look at FIX and the related FAST. These are where big players are going.

If you want to see how others do this sort of thing, you could have a look at opentick :: home The protocol they use is open and the specifications are published on the web site.

 

[RichieE]

If you are not familiar with sockets you really need to learn some more about network programming. It is basic and required knowledge. Stevens book on Unix network programming is recommended - even if you are working on windows.

Thanks for that, I'll be off to the bookshop today!

I would suggest that SOAP is not the tool for this job. XML is too "heavy" for this sort of thing. In any case SOAP is mostly an RPC type mechanism, which is not really what you are trying to do.

Great, I've been looking for some indication as to whether SOAP will be lean enough to run a realtime application. It has been surprisingly difficult to find out.

If you want to do this right you might have a look at FIX and the related FAST. These are where big players are going.

I will investigate

If you want to see how others do this sort of thing, you could have a look at opentick :: home The protocol they use is open and the specifications are published on the web site.

I've browsed through the Opentick website but they don't seem to have any open source code for the server side... only the client side. (Unless I've missed something that is...)

 

[Lucky_81]

Hi RichieE,

What are your trade signals in STIRS based on? Interested to see what you come up with.

 

[dcraig1]

I've browsed through the Opentick website but they don't seem to have any open source code for the server side... only the client side. (Unless I've missed something that is...)

Yes, that's so, but the protocol spec is there somewhere. I haven't read it, but it should give you an idea as to how the messages are assembled and a kick start in designing your own protocol, should you choose to go down that path.

Of course there are many ways of going about this sort of thing, but I would be cautious of "hand assembling" messages. ie bytes 1 - 4 is the bid size etc etc. If you are able to use something that automatically encodes/decodes (or marshalls) the wire protocol is does make life easier.

For simple elegance and high performance, I always liked the XDR protocol as used in SUN's RPC. It is an internet standard (there is an RFC), and the compiler "rpcgen" is available on just about all *nix systems. Unfortunately it is oriented towards C, so is probably not want you want, but is a good example of what I am talking about.

 

[RichieE]

Hi RichieE,

What are your trade signals in STIRS based on? Interested to see what you come up with.

Hi Lucky, we're still working on that part! Backtesting is in progress... I'm working as part of a group of prop traders and the original trade ideas are not mine; therefore I don't feel that it would be right to reveal someone else's work on a public forum. However, I suspect the ideas are not entirely original so a little research will probably uncover very similar concepts.

 

[RichieE]

Yes, that's so, but the protocol spec is there somewhere. I haven't read it, but it should give you an idea as to how the messages are assembled and a kick start in designing your own protocol, should you choose to go down that path.

That was very interesting reading. I'm really grateful for that hint.

Of course there are many ways of going about this sort of thing, but I would be cautious of "hand assembling" messages. ie bytes 1 - 4 is the bid size etc etc.

Why do you say that exactly? Is this solely to remove the admin of keeping a proprietary protocol? Or are there other issues?

If you are able to use something that automatically encodes/decodes (or marshalls) the wire protocol is does make life easier.

That would be perfect. Right now my dream scenario would be to find an ActiveX Control or class on the internet that I could incorporate into my software. This would act as a "tunnel" between my 'server' and my 'client' PCs. I simply want to call a method on the server passing a price data and a timestamp. This will then raise an event in the client PCs which will process that price and timestamp.

For simple elegance and high performance, I always liked the XDR protocol as used in SUN's RPC. It is an internet standard (there is an RFC), and the compiler "rpcgen" is available on just about all *nix systems. Unfortunately it is oriented towards C, so is probably not want you want, but is a good example of what I am talking about.

I've just taken a look at this and it does seem to delve into the fine mechanics... surely there must be a premade 'tunnel' plugin code that I could 'nab'? Do you think I will be wasting my time searching for such a tool?