Overhauled DTS representation of Frictionless data structures. #112
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeff-cohere
added
JDP
|
|
No great hurry on this one, @ialarmedalien . I've got to set up dev/prod DTS instances in Spin. I want to battle-test this more before deploying it with dependent services like Ken's IMG -> KBase push. |
This commit brings in the official Golang module for Frictionless data packages and resources. It's a bit of a mess, given how pervasive these data structures are in the DTS, but most of the work is done.
jeff-cohere
force-pushed
the
frictionless-overhaul
branch
from
c664fa9 to
c1adb08
Compare
ialarmedalien
approved these changes
Mar 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR discards our homemade representation of the Frictionless data structures and adopts the official Go Frictionless DataPackage module.
Aside from letting us stash stuff more flexibly in data packages (manifests) and data resources (loadable/downloadable file-level metadata), it validates these types and produces JSON-sensible descriptors for consumption by clients.
The DTS deals mainly in descriptors. The
datapackage.Packageanddatapackage.Resourcetypes are reserved for use by Go clients, but the validation ensures that such things can be properly constructed as needed.