Handling pre-flight requests directly

[scaturigo]

Some endpoints require pre-flight requests. Can iBind handle these? I'm currently implementing a solution like this :

 response = client.live_marketdata_snapshot(undConid)
    ,["31"])
    while "31" not in response.data[0]:
        sleep(0.1)
        print("not ready ...")
        response = client.live_marketdata_snapshot(undConid) ,["31"])

Is this the recommended approach?

[Voyz]

We could, yeah. I can implement it, it shouldn't be too much work. Do you think you could help out prepping a list of endpoints that need pre-flight requests?

As for what's recommended - I suggest you ask IBKR support about this, I don't know how their server side is implemented.

[scaturigo]

Ok. After looking the code, The pre-flight checks for the /iserver/marketdata/snapshot endpoint are present in the higher-level live_marketdata_snapshot_by_symbol method, which takes a symbol as input. However, these checks are absent in the lower-level live_marketdata_snapshot method, which directly calls the endpoint. For improved consistency and to ensure the pre-flight logic is always executed, it's better to move it to live_marketdata_snapshot. As for listing endpoint, I see three endpoints with pre-flight logic :

1. GET /iserver/marketdata/snapshot
2. GET /iserver/account/orders
3. GET /iserver/contract/{{ conid }}/algos

[Voyz]

Thanks for these observations.

Right, I'm trying to think how to implement this correctly. I remember that I've considered doing the pre-flights in library before, but decided not to because of the following limitations. We could:

1. Do what you suggest initially, check if a field is present in the response. For example, we could check the first field in the fields parameter.
   • However, there's an issue with this. Sometimes that endpoint will return incomplete data, in that only some fields will be made available.
   • I've seen it before myself, that out of 10 fields specified, 1 or 2 are never returned. Calling the endpoint repeatedly didn't help.
   • If that's the case, we'd gamble that the value we'd be checking for would be in the batch that does get returned.
2. Check for any data being available.
   • But sometimes some fields do take longer to propagate. There may be a following case when trying to query 5 different fields:
     • response one - contains no fields, pre-flight
     • response two and three - contains fields 1, 2 and 5
     • response four and onwards - contains fields 1, 2, 3 and 5
   • As per the previous point, field 4 is never returned; field 3 gets returned only after the fourth request

Here lies the issue: If we communicate to the user that we take care of pre-flights, they may not be inclined to continue calling the endpoint in order to collect more data, which may be advisable in some cases. I've seen similar issue with live orders pre-flights.

I'm on the fence, since I do think it would be a good idea to automate things if possible. Not doing it however, would cause each user to read about the pre-flights individually, and through hands-on trial and error discover how the data is being fed back to them - allowing them to adjust their system accordingly.

Thoughts?

[scaturigo]

That's a very interesting point. I think the responsibility for pre-flight requests should fall to the developer, not be handled automatically by iBind. As you pointed out, we don't always know how the server works, and too many requests could get us IP-banned by IBKR. While iBind's automatic pre-flight handling is very, very convenient, it might be better to disable it by default, with an option to enable it when needed.

[Voyz]

Right, thanks for sharing your view on this. I think I agree, we shouldn't take care of this for users.

I updated the WiKi to explain the pre-flight situation better. I think this is a responsible solution to the problem. Please have a look and let me know if anything could be modified there: https://github.com/Voyz/ibind/wiki/Advanced-REST#-pre-flight-requests