TCP Stream v2
Spire Maritime’s TCP Stream provides AIS Data in NMEA format like shown in the next section. This service has been re-developed to scale for performance with the continually expanding data volumes of the Spire AIS service.
Our data volumes have grown around 300% in the last 2 years and our Satellite AIS data volumes have grown 30% in just the first 3 months of 2021.
The NMEA message encoding format
The National Marine Electronics Association (NMEA) controls the standard for encoded AIS messages.
The Spire Maritime TCP Feed sends encoded AIS messages in the NMEA 0183 v4.0 format with a timestamp
TAG block as a prefix.
Some example messages from the Spire Maritime TCP Feed are below:
\c:1503079500*55\!AIVDM,1,1,,B,C6:b0Kh09b3t1K4ChsS2FK008NL>`[email protected],0*50 \c:1503079517*53\!AIVDM,1,1,,B,16:Vk1h00g8O=vRBDhNp0nKp0000,0*40 \c:1503079517*53\!AIVDM,1,1,,B,18155hh00u0DEU`[email protected]@D,0*60 \c:1503079517*53\!AIVDM,1,1,,A,[email protected]@g?b`7mL0,0*55 \c:1503079517*53\!AIVDM,2,1,9,A,[email protected]<[email protected]:53kThQDQh0,0*48 \c:1503079517*53\!AIVDM,2,2,9,A,00000000000,2*2D \c:1503079516*52\!AIVDM,1,1,,A,[email protected],0*1A \c:1503079516*52\!AIVDM,1,1,,B,B3mISo000H;wsB8SetMnww`5oP06,0*7C \c:1503079517*53\!AIVDM,2,1,0,B,[email protected]<[email protected]`888,0*1B
TCP server behavior
AIS Message Delivery
AIS messages are delivered in sentences. Each sentence is separated by one “line feed” (
LF / “
AIS messages that comprise of multiple sentences will stream these adjacent sentences in order.
Handling of interrupted connections
If your TCP client gets briefly disabled for any reason, our TCP server uses a cursor to keep track of the last successfully sent AIS message. Once your client reconnects, the cursor indicates the last received message in the stream so no messages get skipped.
If your TCP client is disconnected for an extended period of time (several days, for instance), upon reconnection, it will begin streaming AIS messages that had been published within the past three hours.
Using the TCP stream
Connection to the TCP server
To be able to use the Spire AIS TCP Service you will need an authorization token, which is used to authenticate the service.
For TCP connections, client systems must open a TCP socket connection to the Spire service on the Service Address :
streamingv2.ais.spire.comusing Port number:
To initiate the stream of messages, and initial authentication string must be sent as below:
Note: Depending on which system you are using, you might want to send a new line ‘
\n’ to make sure that the token is sent in one piece directly.
If connection is successful then the stream of AIS data in NMEA format will start as shown in the example below. This can also be used to test the connectivity on your side.
nc streamingv2.ais.spire.com 56784 A|T|USER_TOKEN_HERE
First 3 results of AIS messages being returned
\s:terrestrial,c:1615991449*52\!AIVDM,1,1,,B,B6:bqEh00B1qq7SBon1Tp0HT0000,0*38 \s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,B,133<6J`P00v:QILEV0=v4?wV25ip,0*20 \s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,A,83aDqqhj2d<dd=v<[email protected],0*50
Once your connection has been established, AIS messages should begin streaming into your client.
Please note that it can take up to 60 seconds for the TCP server to start up a new stream after sending your token.
TCP stream IP addresses
The TCP addresses used for streamingv2.ais.spire.com are
Confirmed by the network test shown below:
; <<>> DiG 9.10.6 <<>> streamingv2.ais.spire.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49143 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;streamingv2.ais.spire.com. IN A ;; ANSWER SECTION: streamingv2.ais.spire.com. 24 IN A 220.127.116.11 streamingv2.ais.spire.com. 24 IN A 18.104.22.168 streamingv2.ais.spire.com. 24 IN A 22.214.171.124 ;; Query time: 10 msec ;; SERVER: 126.96.36.199#53(188.8.131.52) ;; WHEN: Tue Apr 26 16:57:45 BST 2022 ;; MSG SIZE rcvd: 102
Matches the elastic IP address associated with the network load balancer for TCP.
Confirmed 26-April-2022 as having not changed since December 2021 when fixed networking addresses were configured for the service.
Recommended reconnection strategy
The TCP server should keep your connection alive indefinitely and send keep-alives every 15 minutes to your client, however, exceptions still happen. Occasionally there might still be instances of unexpected disconnects to the TCP feed. In these cases, you can simply reconnect, re-authenticate with your token and the data stream will resume.
In order to minimize loss of data and impact on latency, we recommend setting up a reconnection method that automatically acts within an hour of any unexpected disconnect.
Interruptions continue where left off
The behaviour of the TCP service is to provide a continuous stream of AIS messages in the order that they are received by Spire systems. If the service stops, due to disconnection or client system problems, then if re-connecting within 7 days from disconnection, the stream of data will continue from the time of disconnection and no data will be missed.
If the service is disconnected for more multiple hours or days then it can cause a large backlog of now old data to be streamed before the service catches up to current time and entirely new messages are received.
If the service is disconnected for more than 7 days then data will be streamed from the current time and only new messages are received. In this situation please contact Spire Maritime sales engineering to request a backfill of historical data that has been missed.
Handling connection lag
If your system is experiencing connection lag (ie. the data you are receiving is well behind real-time results as indicated by the unix timestamp [
c:]), we recommend reviewing your system to see where scaling is necessary to cope with the increased data volume and speed from the TCP V2 feed.
Once the system has been adjusted, refer to the “Updating the Service Checkpoint” section to set the checkpoint to a specified time or to the latest message time (real-time).
Updating the service checkpoint
In the V2 TCP stream service it is possible to update the service checkpoint that specifies where in the stream of data service will continue from.
Setting checkpoint to a specified time
To reset your TCP stream service to start from a particular point in time, pass the authentication token string as normal followed by a date time string in the format
Note the time must be in UTC (zulu time).
Example authentication to set the TCP stream checkpoint to a given time
Note the checkpoint can be set up to 7 days in the past and times most be specified in UTC.
Example of the service checkpoint reset request after which the connection is reset.
nc streamingv2.ais.spire.com 56784
To stream data, connection must be established again.
Setting checkpoint to the latest message time
To reset your TCP stream service to start from a the time of the most recent message in your service, pass the authentication token string as normal followed by the label
Example authentication to set the TCP stream checkpoint to the latest message time
Example of the service checkpoint reset request after which the connection is reset.
nc streamingv2.ais.spire.com 56784 A|T|USER_TOKEN_HERE|resetToLatest
To stream data, connection must be established again.
Forced reconnect after updating the service checkpoint
After setting a checkpoint timestamp, or the checkpoint reset label
resetToLatest, the TCP service will update the checkpoint and disconnect.
You will need to connect again and pass auth string as usual `
A|T|` to get the data from the new checkpoint.
AIS message 5 is commonly received in multiple parts, generally 2 message sentences. Spire Maritime AIS TCP Stream v2 provides grouping tags as shown below. Optionally for users who require it, this feature may be disabled upon request to Spire Maritime sales engineering.
Example of a multipart AIS message 5 showing the NMEA compliant group tag
\g:1-2-1,s:terrestrial,c:1615816351*11\!AIVDM,2,1,1,A,[email protected][email protected]>[email protected]:140Ht00000000000,0*3B \g:2-2-1*6C\!AIVDM,2,2,1,A,00000000000,2*25 \g:1-2-2,s:terrestrial,c:1615816351*12\!AIVDM,2,1,2,B,543Kbi000000DpEJ220M>oS8000000000000001S3HJ366ShN6kRBkk0AE`8,0*2E \g:2-2-2*6F\!AIVDM,2,2,2,B,88888888880,2*25 \g:1-2-3,s:terrestrial,c:1615816351*13\!AIVDM,2,1,3,B,53mA9<42=Ti`hm<B220<thu:[email protected]=V2222220t3c3BB6iWNACSklk88888,0*3F \g:2-2-3*6E\!AIVDM,2,2,3,B,88888888880,2*24 \g:1-2-4,s:terrestrial,c:1615816351*14\!AIVDM,2,1,4,A,[email protected]>[email protected]@[email protected],0*7D \g:2-2-4*69\!AIVDM,2,2,4,A,00000000000,2*20 \g:1-2-5,s:terrestrial,c:1615816351*15\!AIVDM,2,1,5,A,544alp400001DU`6220<[email protected]`[email protected]`PqV20l1HB53800094iEPDm3l3k,0*4E \g:2-2-5*68\!AIVDM,2,2,5,A,88888888880,2*21
Sample TCP Stream results
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,A,342O;[email protected]@[email protected];00>@<,0*34 \s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,[email protected]>46vv2>@<,0*3E \s:terrestrial,c:1615816309*5F\!AIVDM,1,1,,B,[email protected]@hw=sO9;R0p:f,0*55 \s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13aFdSOP0OPO::[email protected]>`<,0*30 \g:1-2-9,s:terrestrial,c:1615816351*19\!AIVDM,2,1,9,B,53c4vr82<ciI0D<N220DhU<[email protected][email protected]<[email protected],0*16 \g:2-2-9*64\!AIVDM,2,2,9,B,R5C38888880,2*39 \g:1-2-10,s:terrestrial,c:1615816351*21\!AIVDM,2,1,0,B,53c4vr82<ciI0D<N220DhU<[email protected][email protected]<[email protected],0*1F \g:2-2-10*5C\!AIVDM,2,2,0,B,R5C38888880,2*30 \s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,[email protected]:v0D10,0*1B \s:dynamic,c:1615815280*4C\!AIVDM,1,1,,A,B3:GbB006B=KfIU;cUAfcwP00000,0*2C \s:dynamic,c:1615815299*44\!AIVDM,1,1,,A,15D2E4?P005gV8F7OW``IVEp0000,0*35 \s:dynamic,c:1615815447*41\!AIVDM,1,1,,A,14`Ut8gP00Ll``IjiVgu?a7p0000,0*5A \s:dynamic,c:1615815539*49\!AIVDM,1,1,,A,17liV?OP008<IrWv5qMtg?wp0000,0*6F \s:dynamic,c:1615815557*41\!AIVDM,1,1,,A,13:1wR?P301tHkNRF4wIC7Op0000,0*1B \s:44405,c:1615815822*0A\!AIVDM,1,1,,A,14eG;[email protected];[email protected]>[email protected],0*2B \s:44405,c:1615815717*03\!AIVDM,1,1,,B,4030p<QvDoeainQABdNN8=7005bT,0*66 \s:44405,c:1615815943*0C\!AIVDM,1,1,,B,14eHUbsP1>o<P8PL6Vt2cww:08EW,0*54 \s:44405,c:1615815898*0B\!AIVDM,1,1,,B,1815;0001lD=tTTM32?SB2eV05bh,0*1F \s:44405,c:1615815976*0A\!AIVDM,1,1,,B,B4eGR9000=fSWF7?;aoQ0c4QnDNr,0*7F \s:44405,c:1615815977*0B\!AIVDM,1,1,,B,14QSvp7000l96t8NvGnto`:F0<0B,0*64
Understanding the AIS message stream format
- Contains 3 parts in a
Xis the fragment number
Yis the count of message fragments
Zis the sequence number that joins the fragments together
- Indicates the AIS source of the received AIS message. Values are:
terrestrialfor AIS received from terrestrial, land based, AIS receivers.
dynamicfor AIS received from dynamic AIS, vessel based, AIS receivers.
<satellite id>for messages received from Spire satellites.
- Note: satellite id will be the NORAD ID of the satellite or, if the NORAD ID is not yet confirmed, the Spire satellite name
- Message timestamp and tags checksum
- Message header
- Message sentence count
1 or 2
- Message sentence number
- Message number
- AIS channel
- (the rest of the message record)
- Encoded AIS data and checksum
Example connection Python code
A simple python script for connecting to the Spire TCP service is available for testing.
It can be called as follows:
python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streamingv2.ais.spire.com 56784 2 <CLIENT AUTHENTICATION TOKEN>
Example script call with a dummy token
python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streamingv2.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8
python3 call_spire_tcp_feed_parameters.py t est_tcp.txt 100 streaming.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgRE VNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8 Program Start Time: 2020-07-22T09:55:43+02:00 server set to parameter value: streaming.ais.spire.com Run TCP client for 2 minutes Debug set to default of False - False Parameter Debug = : False Using API Token from parameters Using API Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEi fSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8 About to try connecting to: Server: streaming.ais.spire.com on port: 56784 Writing text data stream to file test_tcp.txt20200722-075543.txt Writing nmea data stream to file test_tcp.txt20200722-075543_nmea.txt Setting job message limit : 100 Connecting.. program has run for 0.0 minutes. 20200722-075546 Last Minute captured 1 records. Stopping as max_records number reached: 100 Stopping as max_records number reached: 100 Stopped after receiving 100 messages 100 API Processing ran from 20200722-075543 to 20200722-095546 received 100 lines of results Program run from 20200722-075543 to 20200722-095546 took (0.0, 2.779322) minutes Start time as unix time: 1595397343.0 End time as unix time: 1595397346.0 Elapsed time in seconds: 3 wc -l test_tcp.txt20200722-075543_nmea.txt 100 test_tcp.txt20200722-075543_nmea.txt
Showing the captured results
head -5 test_tcp.txt20200722-075543_nmea.txt 20200722-075546,b'\\c:1595382932*50\\!AIVDM,1,1,,B,13Hj;iPP00P7Am0M1`cN4?w02>`<,0*01\r' 20200722-075546,b'\\c:1595382943*56\\!AIVDM,1,1,,A,33QFvD500007=4nM1a0raF5F0>`<,0*3F\r' 20200722-075546,b'\\c:1595382957*53\\!AIVDM,1,1,,A,B3HjFSP0Fh1ba>7A4HFmwwtUoP06,0*1B\r' 20200722-075546,b'\\c:1595382966*51\\!AIVDM,1,1,,B,B3`erch0081Otk7D0MwQ3wS1nE6b,0*00\r' 20200722-075546,b'\\c:1595382969*5E\\!AIVDM,1,1,,B,[email protected]?vB2>`<,0*05\r'
You will need a demo or client authentication token to use this example script.
Not a customer yet? Reach out to our team to request a trial token.
Enhancements over v1
The new TCP service is compatible with the old TCP service but there are multiple enhancements to consider.
Additional Terrestrial AIS
Spire recently established a 3rd partner for provision of terrestrial AIS.
Data from this new partner and any future new partners will only be integrated into the new system.
Swapping to the new TCP stream will bring an immediate increase of about 20% in daily terrestrial AIS messages, a noted uplift in the US Gulf area and 3% uplift in the number of vessels reported by terrestrial AIS each day.
Additional checkpoint controls
The v2 TCP Stream allows clients to set the checkpoint on their service to control the time from which data is streamed after the next re-connection. Read more about this in the “Understanding TCP server behavior” section.
The v2 TCP Stream runs on a new scalable IT infrastructure, designed to cope with the continually increasing AIS data volumes.
This has proven to perform much better than the old TCP, especially when a client catches up on several hours or days of data and this can now be done in minutes compared to hours on the old service.