SpaceDB: Querying a Million Space Objects

A local, queryable database of near-Earth objects and asteroids.

SpaceDB: Querying a Million Space Objects

I wanted a fast local way to ask about the small-body solar system. Near-Earth objects, asteroids and close approaches are all public data from NASA’s Jet Propulsion Laboratory, and I did not want to call their APIs for every question. So I built SpaceDB. It loads JPL’s public small-body and close-approach datasets into one database I can query in milliseconds. This post shows what it holds, how to query it, and what to know about the source data before you trust a number.

What SpaceDB gives you

One place to ask questions across several JPL datasets. The raw feeds come from the small-body database (SBDB), NHATS, Sentry and the close-approach tables. They arrive as JSON, I normalize them into consistent columns, and I load them into the database.

The database holds about 1.5 million object records. There is also a 3D orbit viewer built on spacekit.

Once the data is local and indexed, exploratory questions return at once. That changes how I work. I ask five small questions where I would have written one careful one.

An example query

The API is mine, not JPL’s. This request asks for near-Earth objects larger than 500 meters, soonest approach first:

GET /api/objects?neo=true&diameter_min=0.5&sort=close_approach_date

The diameter_min parameter is in kilometers, so 0.5 means 500 meters. The neo=true filter keeps near-Earth objects only, and sort=close_approach_date puts the nearest date first. Expected shape of the answer: a JSON list of objects, each with its designation, diameter and next approach.

The source is on GitLab.

Where the data comes from

JPL publishes two APIs that feed most of this. Reading their documentation answers most questions I would otherwise guess about.

The small-body query API is GET https://ssd-api.jpl.nasa.gov/sbdb_query.api. It takes fields, sort, limit, limit-from, sb-kind and sb-class. Field names are case sensitive. A documented example asks for the orbital elements of Atira near-Earth asteroids:

https://ssd-api.jpl.nasa.gov/sbdb_query.api?fields=full_name,epoch,e,a,q,i,om,w&sb-class=IEO

The close-approach API is GET https://ssd-api.jpl.nasa.gov/cad.api. It takes date-min, date-max, dist-max and body. The distance is in au by default, and it accepts forms such as 10LD for ten lunar distances. It returns fields including des, orbit_id, jd, cd, dist, dist_min, dist_max, v_rel, v_inf, t_sigma_f and h. Check that the response’s signature.version reads 1.5 before you rely on a parser. JPL documents no rate limit for either API, which is one more reason to fetch once and query the local copy.

Time is TDB, and it is easy to miss

The jd and cd fields in the close-approach data are in TDB, Barycentric Dynamical Time, a time scale used for ephemerides. It is not UTC. The gap is on the order of a minute or two, so it rarely matters for a planning question and does matter if you compare against a clock or a published UTC time.

A column named close_approach_date hides that. When you load JPL’s values unconverted, name the column for the scale it holds, such as close_approach_tdb, so nobody builds on a wrong assumption.

Identifiers

Objects have several identifiers, and they do not mean the same thing. Use spkid or pdes when you need a key that holds across datasets. The id field in the small-body data is internal to JPL’s own database. A join on it will work until it does not.

A few questions to ask first

Which near-Earth objects larger than 500 meters have the next closest approaches? That is the query above.

How many objects belong to each orbit class? Group by class and count. The classes are JPL’s own codes, and IEO is Atira.

Which objects pass within 10 lunar distances in a given year? That is a close-approach filter with a distance under about 0.0257 au. The arithmetic: one lunar distance is 384,400 km and one au is 149,597,870.7 km, so ten lunar distances is 3,844,000 divided by 149,597,870.7, about 0.0257 au.

Each one returns in a fraction of the time a round trip to JPL would take, and that is why a local copy is worth its upkeep. Keeping it fresh is the cost. The local data is only as current as the last load.

A query tool on top

I put Metabase in front of the database so I can filter and chart without writing a script each time. The setup is in Astronomy Querying for Space Objects. It runs in Docker on port 3000, and it is a good way to explore the tables before you write API calls.

Pairing with AstroNN

The other half of the reason is the machine learning work. Fast access to orbital elements and approach records feeds the astroNN experiments I describe in AstroNN + TensorFlow on Apple Silicon: A Setup That Works. A model needs many rows quickly, and a local table gives it that.

What I would check next

Three limits are worth knowing about before you build on any copy of this data.

The data is a snapshot. Orbits of small bodies are refined as new observations arrive, so an element set from the last load can change on the next one. Keep the epoch column with the elements and read it.

Close-approach distances have uncertainty. The cad.api response carries dist_min and dist_max, which bound the distance, and t_sigma_f, the uncertainty of the approach time. Ignoring them turns a range into a false point.

Absolute magnitude h is not a diameter. A diameter is derived from it with an assumed albedo, unless a measured one exists. When you filter on size, ask where the diameter column comes from.

Subscribe to Just another Geek

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe