Skip to content

v0.3.0

Choose a tag to compare

@doctrino doctrino released this 08 Oct 10:14
4c108bd

cdf

Added

  • Added support for device code login flow.
  • Support for CogniteFiles.
  • Docker release of the Toolkit.
  • Support for Hosted Extractors.
  • If there are multiple cognite_toolkit_service_principals, the cdf auth verify will offer to delete duplicates.
  • Dump for Assets and TimeSeries with cdf dump asset and cdf dump timeseries commands.
  • The new cdf modules add subcommand lets users add modules to an existing modules directory.
  • Added support for resource type Sequence in the classic folder.
  • Made the config.[env].yaml optional when running the `cdf build command.
  • Loaders for resource types Assets, 3DModel, and Robotic (Map, DataPostProcessing, RobotCapability,
    Frame, and Location). These loaders were earlier available as feature preview.
  • Support for LocationFilter in the locations folder.
  • Command cdf repo init to initialize the repository with .gitignore, .env, and if the user is on GitHub, creates default workflows for running Toolkit in CI/CD.
  • Command cdf modules list to list all modules.
  • Command cdf modules init to setup modules. This is interactive and will guide you through selecting the
    modules you want to use. You can bypass the interactive mode by using the --all flag. which
    will select all modules, similar to the previous cdf-tk init command.
  • Command cdf modules upgrade to upgrade all modules automatically.

Changed

  • Running cdf dump datamodels now dumps containers and views into separate folders.
  • When running deploy --dry-run, the Toolkit will not require write access to the CDF project.
  • The Toolkit no longer gives a warning if it cannot import Cognite Function code when executing the cdf build
    command. This is to separate the build and deploying of artifacts from validating/debugging the function code.
    Validation of the function code is expected to be handled by cdf run function local.
  • [BREAKING] The resource folder timeseries_datapoints is removed and csv and parquet files with
    datapoints are now expected to be in the timeseries folder.
  • The dump of data models has changed interface, now the data model id is optionally given as positional instead
    of flags. If now data model is given, the user will be prompted to select one.
  • Added flag --modules/-m to select which modules to build in the cdf build command.
  • The cdf build command no longer requires config.[env].yaml to be present. If it is not present, the Toolkit
    will use the default values for the environment.
  • [BREAKING] The command cdf auth verify has been split into cdf auth init and cdf auth verify. The init command
    is used to initialize the auth parameters, and the verify command is used to verify that required privileges are
    set. The init command will by default run the verify command after the initialization unless a --no-verify
    flag is set. In addition, the two commands have been reworked to be more user-friendly. They are now interactive
    (no longer requires a --interactive flag) and have no longer supports passing in a custom Group file. Instead,
    they are intended to only set up and verify a service principal for the Toolkit.
  • The WorkflowTrigger config files now supports object (dict) as a valid type for the
    data field. This will automatically be converted to a json string when deploying the trigger.
  • In the commands build and pull, modules upgrade and modules list, organization-dir is now an
    optional argument --organization-dir and -o instead of positional argument. This is to have consistent
    behavior with other commands.
  • The resource FunctionSchedule is not uniquely identified by name instead of cronExpression by the Toolkit.
    This enables multiple schedules with the same cron expression for the same function.
  • The Toolkit no longer gives warnings for naming conventions that are not followed.
  • [BREAKING] The resource Label is now in the resource folder classic and not in the labels folder.

Fixed

  • Uploading a file will no longer overwrite the FileMetadata.
  • Running cdf auth verify now prompts the user to create the cognite_toolkit_service_principal group if it does not exist.
  • In the cdf build command, version variables (DataModel, View, Transformations) are now always read as
    string and not integer. This ensures no false positive warnings for missing dependencies.
  • In the cdf deploy/clean command, if a transformation is writing to a DataModel or a View the version is
    now always read as a string. Before if it was for example, 3_0_0 it would be read as 300 and cause
    the deployment of the transformation to fail.
  • If you removed metadata in any resource file, the Toolkit would not update this change in CDF. This is now fixed.
  • The config value of a ExtractionPipelineConfig is now correctly parsed as a string. Before it was parsed as YAML,
    typically an object/dict, which caused loss of information. This is because
    yaml.safe_dump(yaml.safe_load(config)) != config as, for example, special YAML tags are lost.
  • The field default_organization_dir was not read in the cdf.toml file. This is now fixed.
  • If the version of a DataModel or View was set to 1_0_0 in the resource file, Toolkit would send
    it as 100 to the API. This is now fixed.
  • Groups without metadata no longer triggers redeploy when running cdf deploy
  • When running any command, the terminal would print warnings from the cognite-sdk. This is now fixed.

Removed

  • CSV files in the timeseries_datapoins folder with the string timeshift_ in the index will no longer
    be timeshifted to today when running cdf build
  • FileMetadata pattern $FILENAME is no longer supports prefix and suffix in the name parameter. This is to
    simplify the pattern.
  • [BREAKING] The command cdf describe is removed. This functionality was not used and thus removing it to simplify
    the Toolkit and to focus on the core functionality.
  • [BREAKING] Support for api parameters in Node resource is removed.
  • Toolkit no longer support Python 3.9.
  • The --interactive flag from the cdf deploy and cdf clean commands.
  • The shared flags --verbose. This has been deprecated and has been replaced by --verbose on each individual
    command. For example, before you could write cdf --verbose build --env dev, now you should write
    cdf build --env dev --verbose.
  • [BREAKING] The command cdf-tk init it now replaced by cdf repo init and cdf modules init.

Templates

Added

  • CogniteCore full and minimum extension templates in quickstart and example packages.
  • Added bootcamp modules ice_cream_api and oee.
  • New modules cognite_modules/inrobot.

Details

What's Changed

  • Fix locale for git in devcontainer by @gregertw in #852
  • [CDF-22327] โš™ Move asset, robotics and 3D loaders by @doctrino in #853
  • [CDF-22333]๐Ÿ”ฉReplace CogniteClient with ToolkitClient by @doctrino in #854
  • [CDF-22338] ๐Ÿƒ Int versioned Views and Data Models always redeploy by @doctrino in #861
  • [CDF-22328] ๐Ÿ’ฃReplace cdf-tk init with cdf-tk modules init by @doctrino in #855
  • [CDF-22329] โ˜ขDelete manuel migration by @doctrino in #856
  • [CDF-22336] ๐Ÿงฌ Function identifier by @doctrino in #857
  • [CDF-22335] ๐Ÿค Remove naming checks by @doctrino in #858
  • [CDF-22337] ๐Ÿน Tracking modules command by @doctrino in #859
  • [CDF-22349] ๐Ÿฆฎ Introduce Config by @doctrino in #865
  • Location filter support by @ronpal in #833
  • include common folder for inrobot functions by @ana-gheorghe in #868
  • [CDF-22374] ๐ŸฆˆMove Feature Flag into cdf.toml by @doctrino in #872
  • [CDF-22348] ๐ŸซฃGraphQL Support by @doctrino in #864
  • [CDF-22378] ๐Ÿ  Modules list command by @doctrino in #874
  • [CDF-22356] ๐Ÿก Rework run function part 1 by @doctrino in #866
  • New dataclass for packages by @ronpal in #869
  • [CDF-22421] ๐Ÿข Introduce Organization Directory by @doctrino in #877
  • Reference templates by @ronpal in #824
  • Default cdf.toml values by @ronpal in #890
  • ci: fix try ignore by @doctrino in #891
  • [CDF-22207, CDF-22356] ๐Ÿ‹ Running Functions Locally by @doctrino in #878
  • [CDF-22356] ๐Ÿ˜ŒRemove old function app by @doctrino in #879
  • ๐Ÿ›‘ Not check test directory for dependencies. by @doctrino in #905
  • [CDF-22448] ๐Ÿ˜Ž Build errors by @doctrino in #906
  • [CDF-22441] ๐Ÿงฑ Robustify reading feature flags by @doctrino in #908
  • [CDF-22474] ๐Ÿปโ€โ„๏ธ Classic Resource Folder by @doctrino in #907
  • [CDF-22341] ๐ŸชฆRip cognite modules by @ronpal in #896
  • [CDF-22341] ๐Ÿ—๏ธ modules init --all (--clean) copies all files from _builtin_modules to org_dir/modules by @ronpal in #916
  • [CDF-22496] ๐ŸŽธ Instrument feature flag by @doctrino in #915
  • [CDF-22353] ๐ŸŽ› Module metrics by @ronpal in #917
  • [CDF-22493] ๐ŸŒดIntroduce repo Command by @doctrino in #919
  • [CDF-22495] ๐ŸฆญAdd back in skipped tests by @doctrino in #918
  • [CDF-22492] ๐Ÿ‘ทโ€โ™‚๏ธ Fix creation config.dev.yaml by @doctrino in #920
  • [CDF-22457] ๐Ÿคผ Migrations by @doctrino in #921
  • [CDF-22507, CDF-22508, CDF-22509, CDF-22512]๐Ÿ˜Last Minute Fixes by @doctrino in #922
  • better user guide by @ronpal in #924
  • Prepare 0.3.0a1 by @doctrino in #923
  • fix: bump command by @doctrino in #926
  • Prepare release 0.3.0a2 by @doctrino in #927
  • ๐ŸผCI Remove extra jobs by @doctrino in #929
  • friendlier message if no functions are found by @ronpal in #932
  • Changed prompts that were confusing by @ronpal in #933
  • Remove SDK Warnings by @doctrino in #934
  • Fixing location filter by @ronpal in #936
  • [CDF-22358] โ–ถWorkflow trigger loader by @doctrino in #930
  • [CDF-22467] ๐Ÿ™ƒVersion as string by @doctrino in #931
  • [CDF-22525] ๐Ÿฅบ Consistent use of org by @doctrino in #935
  • Prepare release0.3.0a3 by @doctrino in #937
  • ๐Ÿ›fix: pick up default org in cdm.toml by @doctrino in #939
  • [CDF-22528] ๐Ÿ™„ Support dict workflowtrigger by @doctrino in #940
  • [CDF-22527] ๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘Userfriendly message if modules and config not found. by @doctrino in #941
  • [CDF-22526] ๐Ÿคธโ€โ™‚๏ธ Migration Fix by @doctrino in #942
  • [CDF-22467] ๐Ÿ˜• GraphQL Loader version bug by @doctrino in #943
  • [CDF-22538] ๐Ÿฆ›Group without metadata by @doctrino in #946
  • [CDF-22518] ๐Ÿง— GraphQL Parser by @doctrino in #944
  • Prepare 0.3.0a4 by @doctrino in #948
  • [CDF-22331, CDF-22544] ๐ŸงExtraction Pipeline Config parse config as string by @doctrino in #950
  • [CDF-22351] modules add subcommand by @ronpal in #955
  • [CDF-21821, CDF-21860, CDF-22355] Split auth in inti and verify by @doctrino in #956
  • [CDF-22414] ๐Ÿฃ Hosted extractor source loader by @doctrino in #951
  • [CDF-22345] ๐Ÿฅฆ Support Sequences by @doctrino in #958
  • [CDF-22564] ๐Ÿ– Cleanup deploy/clean commands by @doctrino in #959
  • [CDF-22579] ๐Ÿ˜ฎโ€๐Ÿ’จLocation filter with data models by @doctrino in #962
  • Fix location filter dump by @doctrino in #964
  • acl for sequences by @ronpal in #963
  • Prepare 0.3.0a5 by @doctrino in #965
  • [CDF-22563] ๐Ÿง˜โ€โ™‚๏ธFix update by @doctrino in #970
  • add bootcamp initial config for participants by @petreeb in #971
  • [CDF-22565] ๐ŸซตBuild Selected Option by @doctrino in #961
  • [CDF-22420, CDF-22424] adding plugins and updating feature_flags. Tracking use of plugins in MixPanel by @ronpal in #960
  • Prepare 0.3.0a6 by @doctrino in #973
  • [CDF-22632] ๐ŸฆฟTOML Error by @doctrino in #975
  • [CDF-22635] ๐Ÿง‘โ€๐ŸซLocation Testing by @doctrino in #977
  • [CDF-22634] ๐Ÿ› Migration from 0.2.x to 0.3 alpha by @doctrino in #976
  • [CDF-22637] ๐Ÿค Module upgrade verbose by @doctrino in #978
  • Prepare 0.3.0a7 by @doctrino in #991
  • ๐Ÿค— Tiny cleanup by @doctrino in #969
  • [CDF-22641] ๐Ÿ˜ฎโ€๐Ÿ’จNo longer validate import of function in build step by @doctrino in #979
  • [prep CDF-22639] ๐Ÿ˜ State between build and depoy by @doctrino in #980
  • [CDF-22415] ๐Ÿ” Destination loader by @doctrino in #983
  • [CDF-22416] ๐Ÿง‘โ€๐Ÿ”ง Job loader by @doctrino in #984
  • [CDF-22640, CDF-22639] ๐Ÿšข Deploy state by @doctrino in #981
  • [CDF-22589] ๐ŸŽšRework Pull Command by @doctrino in #985
  • [CDF-22626] ๐Ÿ˜Ž Rename Toolkit Group by @doctrino in #993
  • [CDF-22540] ๐Ÿ–Š Unify dump by @doctrino in #986
  • [CDF-22590]๐Ÿชฆ RIP Describe by @doctrino in #990
  • [CDF-22346]๐Ÿ—’ Support CogniteFile by @doctrino in #982
  • [CDF-22656] ๐Ÿคจ Remove NodeAPICall option by @doctrino in #994
  • [CDF-22655] ๐Ÿ˜ Bootcamp special needs by @doctrino in #995
  • [CDF-22352] ๐Ÿ˜ถ Added deprecation warning by @doctrino in #996
  • [CDF-22660] ๐Ÿ™‚โ€โ†•๏ธ Setup plugins by @doctrino in #998
  • [CDF-22659] ๐Ÿ˜ฌ Drop python3.9 support by @doctrino in #997
  • [CDF-22657] ๐Ÿ˜” Handle no .as_write by @doctrino in #999
  • Automatic fix node by @doctrino in #1002
  • Prepare 0.3.0b1 by @doctrino in #1001
  • [CDF-22638] ๐Ÿคค Auto Release by @doctrino in #1000
  • [CDF-22682] ๐ŸŽ Bootcamp to test by @doctrino in #1004
  • [CDF-22683] ๐Ÿง‘โ€โœˆ๏ธAuth Verify by @doctrino in #1005
  • [CDF-22691] ๐Ÿ˜–Int version in Transformations by @doctrino in #1007
  • [CDF-22636] ๐ŸšดGraphQL Loader Fixes by @doctrino in #1006
  • Prepare 0.3.0b2 by @doctrino in #1008
  • ๐Ÿš…Fix autorelase by @doctrino in #1010
  • ๐Ÿ™ƒAdded back transformation loader tests by @doctrino in #1011
  • [CDF-22636] ๐Ÿ˜ฌ GraphQL Parser by @doctrino in #1012
  • [CDF-22417] ๐Ÿ—บSupport hosted extractor mappings by @doctrino in #1014
  • Prepare 0.3.0b3 by @doctrino in #1015
  • Some branding artifacts by @ronpal in #1022
  • fixed bug with performance and quality timeseries by @petreeb in #1026
  • Adding locationfilter example to builtin modules by @ronpal in #1018
  • Prepare 0.3.0b4 by @ronpal in #1029
  • [CDF-22718] ๐ŸŒณ When running deploy --dry-run, the Toolkit will not require write access to the CDF project. by @ronpal in #1031
  • [CDF-22685] ๐Ÿ˜ผ Builder Concept Part 1 by @doctrino in #1017
  • [CDF-22685] ๐Ÿ˜ผ Builder Concept Part 2 by @doctrino in #1032
  • [CDF-22685] ๐Ÿ˜ผ Builder Concept Part 3 by @doctrino in #1033
  • Add authlib dependency and configure OAuth2 authentication by @gregertw in #846
  • [CDF-22685] ๐Ÿ˜ผ Builder Concept Part 4 by @doctrino in #1034
  • [CDF-22745] ๐Ÿ“ฆ feat: Upload Cognite Files by @doctrino in #1036
  • [CDF-22354] ๐Ÿ‹ Docker image by @doctrino in #1035
  • fix by @doctrino in #1043
  • Prepare 0.3.0b5 by @doctrino in #1044
  • Docker Push fix by @doctrino in #1048
  • ๐Ÿ›‘ Stop trying to update docker files by @doctrino in #1057
  • [CDF-22785] ๐Ÿ›ž Expansion pattern for CogniteFile by @doctrino in #1058
  • [CDF-22805, CDF-22806] ๐Ÿ™‚โ€โ†•๏ธ Core Extension Minimum and Full Models by @doctrino in #1061
  • [CDF-22585] ๐Ÿง‘โ€๐Ÿ”ง Repo init create GitHub workflow by @doctrino in #1047
  • ๐Ÿ‘ถSimplify core extension by @doctrino in #1064
  • Module Dependencies and APM files for InRobot by @ronpal in #1060
  • Prepare 0.3.0 by @doctrino in #1066

New Contributors

Full Changelog: v0.2.20...v0.3.0