TCP Stream v2

Spire Maritime’s TCP Stream provides AIS Data in NMEA format like shown in the next section. This service has been re-developed to scale for performance with the continually expanding data volumes of the Spire AIS service.

Our data volumes have grown around 300% in the last 2 years and our Satellite AIS data volumes have grown 30% in just the first 3 months of 2021.

Was this article useful? Yes No

The NMEA message encoding format

The National Marine Electronics Association (NMEA) controls the standard for encoded AIS messages.

The Spire Maritime TCP Feed sends encoded AIS messages in the NMEA 0183 v4.0 format with a timestamp TAG block as a prefix.

Some example messages from the Spire Maritime TCP Feed are below:

\c:1503079500*55\!AIVDM,1,1,,B,C6:b0Kh09b3t1K4ChsS2FK008NL>`2CT@2N000000000S4h8S400,0*50
\c:1503079517*53\!AIVDM,1,1,,B,16:Vk1h00g8O=vRBDhNp0nKp0000,0*40
\c:1503079517*53\!AIVDM,1,1,,B,18155hh00u0DEU`N1F@Bg22R06@D,0*60
\c:1503079517*53\!AIVDM,1,1,,A,83aGFQ@j2ddtMH1b@g?b`7mL0,0*55
\c:1503079517*53\!AIVDM,2,1,9,A,53m@FJ400000hT5<0008E8q@TpF000000000000T2P3425rg0:53kThQDQh0,0*48
\c:1503079517*53\!AIVDM,2,2,9,A,00000000000,2*2D
\c:1503079516*52\!AIVDM,1,1,,A,13oP50Oi420UAtPgp@UPrP1d01,0*1A
\c:1503079516*52\!AIVDM,1,1,,B,B3mISo000H;wsB8SetMnww`5oP06,0*7C
\c:1503079517*53\!AIVDM,2,1,0,B,53aIjwh000010CSK7R04lu8F222222222222221?9@<297?o060@C51D`888,0*1B

A detailed breakdown of the NMEA 0183 v4.00 format can be found in NMEA’s website

Was this article useful? Yes No

TCP server behavior

AIS Message Delivery

AIS messages are delivered in sentences. Each sentence is separated by one “line feed” (LF / “\n“) character.

AIS messages that comprise of multiple sentences will stream these adjacent sentences in order.

Was this article useful? Yes No

Handling of interrupted connections

If your TCP client gets briefly disabled for any reason, our TCP server uses a cursor to keep track of the last successfully sent AIS message. Once your client reconnects, the cursor indicates the last received message in the stream so no messages get skipped.

If your TCP client is disconnected for an extended period of time (several days, for instance), upon reconnection, it will begin streaming AIS messages that had been published within the past three hours.

Was this article useful? Yes No

Using the TCP stream

Connection to the TCP server

To be able to use the Spire AIS TCP Service you will need an authorization token, which is used to authenticate the service.

Connection initiation

For TCP connections, client systems must open a TCP socket connection to the Spire service on the Service Address : streamingv2.ais.spire.comusing Port number: 56784.

To initiate the stream of messages, and initial authentication string must be sent as below:

A|T|<auth_token_goes_here>

Note: Depending on which system you are using, you might want to send a new line ‘\n’ to make sure that the token is sent in one piece directly.

If connection is successful then the stream of AIS data in NMEA format will start as shown in the example below. This can also be used to test the connectivity on your side.

Example:

Service connection

nc streamingv2.ais.spire.com 56784
A|T|USER_TOKEN_HERE

First 3 results of AIS messages being returned

\s:terrestrial,c:1615991449*52\!AIVDM,1,1,,B,B6:bqEh00B1qq7SBon1Tp0HT0000,0*38
\s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,B,133<6J`P00v:QILEV0=v4?wV25ip,0*20
\s:terrestrial,c:1615991451*5B\!AIVDM,1,1,,A,83aDqqhj2d<dd=v<u0I0N@J53900,0*50

Once your connection has been established, AIS messages should begin streaming into your client.

Please note that it can take up to 60 seconds for the TCP server to start up a new stream after sending your token.

Was this article useful? Yes No

TCP stream IP addresses

The TCP addresses used for streamingv2.ais.spire.com are 35.81.96.111, 44.236.83.108 and 52.39.55.110 .

Confirmed by the network test shown below:

; <<>> DiG 9.10.6 <<>> streamingv2.ais.spire.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49143
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;streamingv2.ais.spire.com.    IN    A
;; ANSWER SECTION:
streamingv2.ais.spire.com. 24    IN    A    35.81.96.111
streamingv2.ais.spire.com. 24    IN    A    44.236.83.108
streamingv2.ais.spire.com. 24    IN    A    52.39.55.110
;; Query time: 10 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Dec 12 15:29:40 UTC 2023
;; MSG SIZE  rcvd: 102

Matches the elastic IP address associated with the network load balancer for TCP.

Confirmed 12-December-2023 as having not changed since November 2023 when fixed networking addresses were configured for the service.

Was this article useful? Yes No

Recommended reconnection strategy

The TCP server should keep your connection alive indefinitely and send keep-alives every 15 minutes to your client, however, exceptions still happen. Occasionally there might still be instances of unexpected disconnects to the TCP feed. In these cases, you can simply reconnect, re-authenticate with your token and the data stream will resume.

In order to minimize loss of data and impact on latency, we recommend setting up a reconnection method that automatically acts within an hour of any unexpected disconnect.

Was this article useful? Yes No

Interruptions continue where left off

The behaviour of the TCP service is to provide a continuous stream of AIS messages in the order that they are received by Spire systems. If the service stops, due to disconnection or client system problems, then if re-connecting within 7 days from disconnection, the stream of data will continue from the time of disconnection and no data will be missed.

If the service is disconnected for more multiple hours or days then it can cause a large backlog of now old data to be streamed before the service catches up to current time and entirely new messages are received.

If the service is disconnected for more than 7 days then data will be streamed from the current time and only new messages are received. In this situation please contact Spire Maritime sales engineering to request a backfill of historical data that has been missed.

Was this article useful? Yes No

Handling connection lag

If your system is experiencing connection lag (ie. the data you are receiving is well behind real-time results as indicated by the unix timestamp [c:]), we recommend reviewing your system to see where scaling is necessary to cope with the increased data volume and speed from the TCP V2 feed.

Once the system has been adjusted, refer to the “Updating the Service Checkpoint” section to set the checkpoint to a specified time or to the latest message time (real-time).

Was this article useful? Yes No

Updating the service checkpoint

In the V2 TCP stream service it is possible to update the service checkpoint that specifies where in the stream of data service will continue from.

Was this article useful? Yes No

Setting checkpoint to a specified time

To reset your TCP stream service to start from a particular point in time, pass the authentication token string as normal followed by a date time string in the format YYYY-MM-DDTHH:MM:SS.MMMZ.

Note the time must be in UTC (zulu time).

Example authentication to set the TCP stream checkpoint to a given time

A|T|<your-token>|2021-03-08T11:45:09.840Z`

Note the checkpoint can be set up to 7 days in the past and times most be specified in UTC.

Example of the service checkpoint reset request after which the connection is reset.

nc streamingv2.ais.spire.com 56784 A|T|USER_TOKEN_HERE|2021-03-17T16:45:09.840Z

To stream data, connection must be established again.

Was this article useful? Yes No

Setting checkpoint to the latest message time

To reset your TCP stream service to start from a the time of the most recent message in your service, pass the authentication token string as normal followed by the label resetToLatest.

Example authentication to set the TCP stream checkpoint to the latest message time

A|T|<your-token>|resetToLatest

Example of the service checkpoint reset request after which the connection is reset.

nc streamingv2.ais.spire.com 56784
A|T|USER_TOKEN_HERE|resetToLatest

To stream data, connection must be established again.

Was this article useful? Yes No

Forced reconnect after updating the service checkpoint

After setting a checkpoint timestamp, or the checkpoint reset label resetToLatest, the TCP service will update the checkpoint and disconnect.

You will need to connect again and pass auth string as usual `A|T|` to get the data from the new checkpoint.

Was this article useful? Yes No

Multipart messages

AIS message 5 is commonly received in multiple parts, generally 2 message sentences. Spire Maritime AIS TCP Stream v2 provides grouping tags as shown below. Optionally for users who require it, this feature may be disabled upon request to Spire Maritime sales engineering.

Example of a multipart AIS message 5 showing the NMEA compliant group tag

\g:1-2-1,s:terrestrial,c:1615816351*11\!AIVDM,2,1,1,A,53kiP6023Ps@=5Hh000T@60>p00000000000000l1@E:140Ht00000000000,0*3B
\g:2-2-1*6C\!AIVDM,2,2,1,A,00000000000,2*25
\g:1-2-2,s:terrestrial,c:1615816351*12\!AIVDM,2,1,2,B,543Kbi000000DpEJ220M>oS8000000000000001S3HJ366ShN6kRBkk0AE`8,0*2E
\g:2-2-2*6F\!AIVDM,2,2,2,B,88888888880,2*25
\g:1-2-3,s:terrestrial,c:1615816351*13\!AIVDM,2,1,3,B,53mA9<42=Ti`hm<B220<thu:0H4q@5=V2222220t3c3BB6iWNACSklk88888,0*3F \g:2-2-3*6E\!AIVDM,2,2,3,B,88888888880,2*24 \g:1-2-4,s:terrestrial,c:1615816351*14\!AIVDM,2,1,4,A,53aLlsP0000109U@0008A>10Thu@00000000000j1@63340003@000000000,0*7D
\g:2-2-4*69\!AIVDM,2,2,4,A,00000000000,2*20
\g:1-2-5,s:terrestrial,c:1615816351*15\!AIVDM,2,1,5,A,544alp400001DU`6220<50U@4r1`4@u8u`PqV20l1HB53800094iEPDm3l3k,0*4E
\g:2-2-5*68\!AIVDM,2,2,5,A,88888888880,2*21

Sample TCP Stream results

From May 2023, the source tag for satelite received AIS messages will fixed to the value satellite. Satellite ID may be provided by special arrangement only.

\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,A,342O;g@P@61tAMRF00EK@8;00>@<,0*34
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,144i@9P0002f57bIPLw>46vv2>@<,0*3E
\s:terrestrial,c:1615816309*5F\!AIVDM,1,1,,B,152blF0PADIGC@t@hw=sO9;R0p:f,0*55 
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13aFdSOP0OPO::6MqC@3ugvv2>`<,0*30 
\g:1-2-9,s:terrestrial,c:1615816351*19\!AIVDM,2,1,9,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*16 
\g:2-2-9*64\!AIVDM,2,2,9,B,R5C38888880,2*39 
\g:1-2-10,s:terrestrial,c:1615816351*21\!AIVDM,2,1,0,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*1F
\g:2-2-10*5C\!AIVDM,2,2,0,B,R5C38888880,2*30 \s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13GQ3J?OiewluSJHkUO@HP:v0D10,0*1B \s:dynamic,c:1615815280*4C\!AIVDM,1,1,,A,B3:GbB006B=KfIU;cUAfcwP00000,0*2C
\s:dynamic,c:1615815299*44\!AIVDM,1,1,,A,15D2E4?P005gV8F7OW``IVEp0000,0*35
\s:dynamic,c:1615815447*41\!AIVDM,1,1,,A,14`Ut8gP00Ll``IjiVgu?a7p0000,0*5A 
\s:dynamic,c:1615815539*49\!AIVDM,1,1,,A,17liV?OP008<IrWv5qMtg?wp0000,0*6F 
\s:dynamic,c:1615815557*41\!AIVDM,1,1,,A,13:1wR?P301tHkNRF4wIC7Op0000,0*1B 
\s:satellite,c:1615815822*0A\!AIVDM,1,1,,A,14eG;oh2@0o;eDtL@>37Sn140@Do,0*2B 
\s:satellite,c:1615815717*03\!AIVDM,1,1,,B,4030p<QvDoeainQABdNN8=7005bT,0*66 
\s:satellite,c:1615815943*0C\!AIVDM,1,1,,B,14eHUbsP1>o<P8PL6Vt2cww:08EW,0*54 
\s:satellite,c:1615815898*0B\!AIVDM,1,1,,B,1815;0001lD=tTTM32?SB2eV05bh,0*1F 
\s:satellite,c:1615815976*0A\!AIVDM,1,1,,B,B4eGR9000=fSWF7?;aoQ0c4QnDNr,0*7F 
\s:satellite,c:1615815977*0B\!AIVDM,1,1,,B,14QSvp7000l96t8NvGnto`:F0<0B,0*64

Prior to May 2023 clients could receive the NORAD satellite ID in the source tag for satelite received AIS messages. An example of this is below

\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,A,342O;g@P@61tAMRF00EK@8;00>@<,0*34
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,144i@9P0002f57bIPLw>46vv2>@<,0*3E
\s:terrestrial,c:1615816309*5F\!AIVDM,1,1,,B,152blF0PADIGC@t@hw=sO9;R0p:f,0*55
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13aFdSOP0OPO::6MqC@3ugvv2>`<,0*30
\g:1-2-9,s:terrestrial,c:1615816351*19\!AIVDM,2,1,9,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*16
\g:2-2-9*64\!AIVDM,2,2,9,B,R5C38888880,2*39
\g:1-2-10,s:terrestrial,c:1615816351*21\!AIVDM,2,1,0,B,53c4vr82<ciI0D<N220DhU<48E@R222222222217=@<99toa0=nQA@TUAiiO,0*1F
\g:2-2-10*5C\!AIVDM,2,2,0,B,R5C38888880,2*30
\s:terrestrial,c:1615816351*52\!AIVDM,1,1,,B,13GQ3J?OiewluSJHkUO@HP:v0D10,0*1B
\s:dynamic,c:1615815280*4C\!AIVDM,1,1,,A,B3:GbB006B=KfIU;cUAfcwP00000,0*2C
\s:dynamic,c:1615815299*44\!AIVDM,1,1,,A,15D2E4?P005gV8F7OW``IVEp0000,0*35
\s:dynamic,c:1615815447*41\!AIVDM,1,1,,A,14`Ut8gP00Ll``IjiVgu?a7p0000,0*5A
\s:dynamic,c:1615815539*49\!AIVDM,1,1,,A,17liV?OP008<IrWv5qMtg?wp0000,0*6F
\s:dynamic,c:1615815557*41\!AIVDM,1,1,,A,13:1wR?P301tHkNRF4wIC7Op0000,0*1B
\s:44405,c:1615815822*0A\!AIVDM,1,1,,A,14eG;oh2@0o;eDtL@>37Sn140@Do,0*2B
\s:44405,c:1615815717*03\!AIVDM,1,1,,B,4030p<QvDoeainQABdNN8=7005bT,0*66
\s:44405,c:1615815943*0C\!AIVDM,1,1,,B,14eHUbsP1>o<P8PL6Vt2cww:08EW,0*54
\s:44405,c:1615815898*0B\!AIVDM,1,1,,B,1815;0001lD=tTTM32?SB2eV05bh,0*1F
\s:44405,c:1615815976*0A\!AIVDM,1,1,,B,B4eGR9000=fSWF7?;aoQ0c4QnDNr,0*7F
\s:44405,c:1615815977*0B\!AIVDM,1,1,,B,14QSvp7000l96t8NvGnto`:F0<0B,0*64

Understanding the AIS message stream format

\g:1-2-5 and \g:2-2-5 group tag: Contains 3 parts in a X-Y-Z format:
X is the fragment number
Y is the count of message fragments
Z is the sequence number that joins the fragments together
\s source tag: Indicates the AIS source of the received AIS message. Values are:
terrestrial for AIS received from terrestrial, land based, AIS receivers.
dynamic for AIS received from dynamic AIS, vessel based, AIS receivers.
from May 2023 the value is satellite for all satellite received AIS messages; Prior to May 2023 some clients had the Spire satellite NORAD ID or satellite name returned for messages received from Spire satellites. This behaviour is now obsoleted
\c:1601972308*58: Message timestamp and tags checksum
!AIVDM: Message header
2: Message sentence count
1 or 2: Message sentence number
5: Message number
A or B: AIS channel
(the rest of the message record): Encoded AIS data and checksum

Was this article useful? Yes No

Example connection Python code

A simple python script for connecting to the Spire TCP service is available for testing.

Download the example script

It can be called as follows:

python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streamingv2.ais.spire.com 56784 2 <CLIENT AUTHENTICATION TOKEN>

Example script call with a dummy token

python call_spire_tcp_feed_parameters.py test_tcp.txt 100 streamingv2.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8

Example response

python3 call_spire_tcp_feed_parameters.py t
est_tcp.txt 100 streaming.ais.spire.com 56784 2 eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgRE
VNTyBET1ZFUiIsInV1aWQiOiI3MTEifSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8
Program Start Time: 2020-07-22T09:55:43+02:00
server set to parameter value: streaming.ais.spire.com
Run TCP client for 2 minutes
Debug set to default of False - False
Parameter Debug = : False
Using API Token from parameters
Using API Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lciI6eyJpZCI6IjcxMSIsIm5hbWUiOiJTcGlyZSBUQ1AgREVNTyBET1ZFUiIsInV1aWQiOiI3MTEi
fSwiaXNzIjoic3BpcmUuY29tIiwiaWF0IjoxNTk1NDA0MTUxfQ.kOiZqAYH2-uY6EJZHLKwuqyMhP763CttvFm8DuQZQW8
About to try connecting to:
Server: streaming.ais.spire.com on port: 56784

Writing text data stream to file test_tcp.txt20200722-075543.txt
Writing nmea data stream to file test_tcp.txt20200722-075543_nmea.txt
Setting job message limit : 100
Connecting..
program has run for 0.0 minutes. 20200722-075546 Last Minute captured 1 records.
Stopping as max_records number reached: 100
Stopping as max_records number reached: 100
Stopped after receiving 100 messages 100
API Processing ran from 20200722-075543 to 20200722-095546 received 100 lines of results

Program run from 20200722-075543 to 20200722-095546 took (0.0, 2.779322) minutes

Start time as unix time: 1595397343.0
End time as unix time: 1595397346.0
Elapsed time in seconds: 3

wc -l test_tcp.txt20200722-075543_nmea.txt
100 test_tcp.txt20200722-075543_nmea.txt

Showing the captured results

head -5 test_tcp.txt20200722-075543_nmea.txt
20200722-075546,b'\\c:1595382932*50\\!AIVDM,1,1,,B,13Hj;iPP00P7Am0M1`cN4?w02>`<,0*01\r'
20200722-075546,b'\\c:1595382943*56\\!AIVDM,1,1,,A,33QFvD500007=4nM1a0raF5F0>`<,0*3F\r'
20200722-075546,b'\\c:1595382957*53\\!AIVDM,1,1,,A,B3HjFSP0Fh1ba>7A4HFmwwtUoP06,0*1B\r'
20200722-075546,b'\\c:1595382966*51\\!AIVDM,1,1,,B,B3`erch0081Otk7D0MwQ3wS1nE6b,0*00\r'
20200722-075546,b'\\c:1595382969*5E\\!AIVDM,1,1,,B,13aFfS@P00P7Dc8M1VSN4?vB2>`<,0*05\r'

You will need a demo or client authentication token to use this example script.

Not a customer yet? Reach out to our team to request a trial token.

Was this article useful? Yes No

Enhancements over v1

The new TCP service is compatible with the old TCP service but there are multiple enhancements to consider.

Was this article useful? Yes No

Additional Terrestrial AIS

Spire recently established a 3rd partner for provision of terrestrial AIS.

Data from this new partner and any future new partners will only be integrated into the new system.

Swapping to the new TCP stream will bring an immediate increase of about 20% in daily terrestrial AIS messages, a noted uplift in the US Gulf area and 3% uplift in the number of vessels reported by terrestrial AIS each day.

Was this article useful? Yes No

Additional NMEA Tags

The NMEA g tag has now been implemented in the new TCP service. It is not used in the current service. So multipart messages (mainly message 5) will be received with the g tag clarifying the 2 parts of the message.

In the current system it is not required because we ensure the 2 message parts are always sent together but for the sake of absolute correctness we have implemented the tag in the new system.

Example of g tag in TCP Stream V2:

\g:1-2-0,s:FM129,c:1611716038*43\!AIVDM,2,1,0,A,59NSD:@2BSk8CD<;N20M8404<F22162JlE;4ScNG3F83FU8888,0*60
\g:2-2-0*6D\!AIVDM,2,2,0,A,88888888880,2*24

If including the g tag in the TCP stream causes you compatibility issues then we have the ability to disable it on a per customer basis.

Was this article useful? Yes No

Additional checkpoint controls

The v2 TCP Stream allows clients to set the checkpoint on their service to control the time from which data is streamed after the next re-connection. Read more about this in the “Understanding TCP server behavior” section.

Was this article useful? Yes No

Performance

The v2 TCP Stream runs on a new scalable IT infrastructure, designed to cope with the continually increasing AIS data volumes.

This has proven to perform much better than the old TCP, especially when a client catches up on several hours or days of data and this can now be done in minutes compared to hours on the old service.

Was this article useful? Yes No