frictionless-ckan-mapper
A library for mapping CKAN metadata <=> Frictionless metadata.
The library has zero dependencies (not even on Data Package libs). You can use it directly or use it for inspiration. Detailed outline of the algorithm is in the docs or you can read the code.
#
Installation- Python: install Python. The library is compatible with both Python 2.7+ and Python 3.3+.
Note: The package is installed as frictionless-ckan-mapper
and then imported as frictionless_ckan_mapper
.
#
Getting started#
CKAN => Frictionless#
Frictionless => CKAN#
ReferenceThis package contains two modules:
frictionless_to_ckan
ckan_to_frictionless
You can import them directly like so:
ckan_to_frictionless
#
resource(ckandict)
#
dataset(ckandict)
#
frictionless_to_ckan
#
resource(fddict)
#
package(fddict)
#
#
Design#
CKAN referenceSummary:
- Class diagram below of key objects (without attributes)
- Objects with their attributes in this spreadsheet: https://docs.google.com/spreadsheets/d/1XdqGTFni5Jfs8AMbcbfsP7m11h9mOHS0eDtUZtqGVSg/edit#gid=1925460244
Source for CKAN metadata structure:
- Dataset (Package): https://docs.ckan.org/en/2.8/api/index.html#ckan.logic.action.create.package_create
Package.as_dict
method: https://github.com/ckan/ckan/blob/2.8/ckan/model/package.py#L195-L223package_show
...
#
Algorithm: CKAN => FrictionlessSee the code in frictionless_ckan_mapper/ckan_to_frictionless.py
#
Algorithm: Frictionless => CKANSee the code in frictionless_ckan_mapper/frictionless_to_ckan.py
#
Developers#
Install the sourceClone the repo:
And install it with pip:
#
Run the testsUse the excellent pytest
suite as follows:
To test under both Python 2 and Python 3 environments, we use tox
. You can run the following command:
Note: Make sure that the necessary Python versions are in your environment PATH
(Python 2.7 and Python 3.6).
#
Building and publishing the packageTo see a list of available commands from the Makefile
, execute:
#
Build the distribution package locally for testing purposesIf a previous build exists, make sure to also remove it before building again:
Then:
Alternatively, this command will accomplish the same to build packages for both Python 2 and Python 3:
#
Test the package at test.pypi.orgThe package will be publicly available at https://test.pypi.org/project/frictionless-ckan-mapper/ and you will be able to pip install
it as usual.
#
Tag a new Git release and publish to the official PyPiMake sure to update the version of the package in the file frictionless_ckan_mapper/VERSION
. Then:
You can quickly review the version to release with make version
, which will print the current version stored in VERSION
.