v0.3.0
cdf
Added
- Added support for device code login flow.
- Support for
CogniteFiles. - Docker release of the Toolkit.
- Support for
Hosted Extractors. - If there are multiple
cognite_toolkit_service_principals, thecdf auth verifywill offer to delete duplicates. - Dump for
AssetsandTimeSerieswithcdf dump assetandcdf dump timeseriescommands. - The new
cdf modules addsubcommand lets users add modules to an existing modules directory. - Added support for resource type
Sequencein theclassicfolder. - Made the
config.[env].yamloptional when running the `cdf build command. - Loaders for resource types
Assets,3DModel, andRobotic(Map,DataPostProcessing,RobotCapability,
Frame, andLocation). These loaders were earlier available as feature preview. - Support for
LocationFilterin thelocationsfolder. - Command
cdf repo initto initialize the repository with.gitignore,.env, and if the user is on GitHub, creates default workflows for running Toolkit in CI/CD. - Command
cdf modules listto list all modules. - Command
cdf modules initto setup modules. This is interactive and will guide you through selecting the
modules you want to use. You can bypass the interactive mode by using the--allflag. which
will select all modules, similar to the previouscdf-tk initcommand. - Command
cdf modules upgradeto upgrade all modules automatically.
Changed
- Running
cdf dump datamodelsnow dumps containers and views into separate folders. - When running
deploy --dry-run, the Toolkit will not require write access to the CDF project. - The Toolkit no longer gives a warning if it cannot import Cognite Function code when executing the
cdf build
command. This is to separate the build and deploying of artifacts from validating/debugging the function code.
Validation of the function code is expected to be handled bycdf run function local. - [BREAKING] The resource folder
timeseries_datapointsis removed andcsvandparquetfiles with
datapoints are now expected to be in thetimeseriesfolder. - The dump of data models has changed interface, now the data model id is optionally given as positional instead
of flags. If now data model is given, the user will be prompted to select one. - Added flag
--modules/-mto select which modules to build in thecdf buildcommand. - The
cdf buildcommand no longer requiresconfig.[env].yamlto be present. If it is not present, the Toolkit
will use the default values for the environment. - [BREAKING] The command
cdf auth verifyhas been split intocdf auth initandcdf auth verify. Theinitcommand
is used to initialize the auth parameters, and theverifycommand is used to verify that required privileges are
set. Theinitcommand will by default run theverifycommand after the initialization unless a--no-verify
flag is set. In addition, the two commands have been reworked to be more user-friendly. They are now interactive
(no longer requires a--interactiveflag) and have no longer supports passing in a custom Group file. Instead,
they are intended to only set up and verify a service principal for the Toolkit. - The
WorkflowTriggerconfig files now supportsobject(dict) as a valid type for the
datafield. This will automatically be converted to ajsonstring when deploying the trigger. - In the commands
buildandpull,modules upgradeandmodules list,organization-diris now an
optional argument--organization-dirand-oinstead of positional argument. This is to have consistent
behavior with other commands. - The resource
FunctionScheduleis not uniquely identified bynameinstead ofcronExpressionby the Toolkit.
This enables multiple schedules with the same cron expression for the same function. - The Toolkit no longer gives warnings for naming conventions that are not followed.
- [BREAKING] The resource
Labelis now in the resource folderclassicand not in thelabelsfolder.
Fixed
- Uploading a file will no longer overwrite the
FileMetadata. - Running
cdf auth verifynow prompts the user to create thecognite_toolkit_service_principalgroup if it does not exist. - In the
cdf buildcommand, version variables (DataModel,View,Transformations) are now always read as
string and not integer. This ensures no false positive warnings for missing dependencies. - In the
cdf deploy/cleancommand, if a transformation is writing to aDataModelor aViewthe version is
now always read as a string. Before if it was for example,3_0_0it would be read as300and cause
the deployment of the transformation to fail. - If you removed
metadatain any resource file, the Toolkit would not update this change in CDF. This is now fixed. - The
configvalue of aExtractionPipelineConfigis now correctly parsed as a string. Before it was parsed as YAML,
typically an object/dict, which caused loss of information. This is because
yaml.safe_dump(yaml.safe_load(config)) != configas, for example, special YAML tags are lost. - The field
default_organization_dirwas not read in thecdf.tomlfile. This is now fixed. - If the version of a
DataModelorViewwas set to1_0_0in the resource file, Toolkit would send
it as100to the API. This is now fixed. - Groups without metadata no longer triggers redeploy when running
cdf deploy - When running any command, the terminal would print warnings from the
cognite-sdk. This is now fixed.
Removed
- CSV files in the
timeseries_datapoinsfolder with the stringtimeshift_in the index will no longer
be timeshifted to today when runningcdf build - FileMetadata pattern
$FILENAMEis no longer supports prefix and suffix in thenameparameter. This is to
simplify the pattern. - [BREAKING] The command
cdf describeis removed. This functionality was not used and thus removing it to simplify
the Toolkit and to focus on the core functionality. - [BREAKING] Support for api parameters in
Noderesource is removed. - Toolkit no longer support Python 3.9.
- The
--interactiveflag from thecdf deployandcdf cleancommands. - The shared flags
--verbose. This has been deprecated and has been replaced by--verboseon each individual
command. For example, before you could writecdf --verbose build --env dev, now you should write
cdf build --env dev --verbose. - [BREAKING] The command
cdf-tk initit now replaced bycdf repo initandcdf modules init.
Templates
Added
- CogniteCore full and minimum extension templates in
quickstartandexamplepackages. - Added
bootcampmodulesice_cream_apiandoee. - New modules
cognite_modules/inrobot.
Details
What's Changed
- Fix locale for git in devcontainer by @gregertw in #852
- [CDF-22327] โ Move asset, robotics and 3D loaders by @doctrino in #853
- [CDF-22333]๐ฉReplace CogniteClient with ToolkitClient by @doctrino in #854
- [CDF-22338] ๐ Int versioned Views and Data Models always redeploy by @doctrino in #861
- [CDF-22328] ๐ฃReplace
cdf-tk initwithcdf-tk modules initby @doctrino in #855 - [CDF-22329] โขDelete manuel migration by @doctrino in #856
- [CDF-22336] ๐งฌ Function identifier by @doctrino in #857
- [CDF-22335] ๐ค Remove naming checks by @doctrino in #858
- [CDF-22337] ๐น Tracking modules command by @doctrino in #859
- [CDF-22349] ๐ฆฎ Introduce Config by @doctrino in #865
- Location filter support by @ronpal in #833
- include common folder for inrobot functions by @ana-gheorghe in #868
- [CDF-22374] ๐ฆMove Feature Flag into cdf.toml by @doctrino in #872
- [CDF-22348] ๐ซฃGraphQL Support by @doctrino in #864
- [CDF-22378] ๐ Modules list command by @doctrino in #874
- [CDF-22356] ๐ก Rework run function part 1 by @doctrino in #866
- New dataclass for packages by @ronpal in #869
- [CDF-22421] ๐ข Introduce Organization Directory by @doctrino in #877
- Reference templates by @ronpal in #824
- Default cdf.toml values by @ronpal in #890
- ci: fix try ignore by @doctrino in #891
- [CDF-22207, CDF-22356] ๐ Running Functions Locally by @doctrino in #878
- [CDF-22356] ๐Remove old function app by @doctrino in #879
- ๐ Not check test directory for dependencies. by @doctrino in #905
- [CDF-22448] ๐ Build errors by @doctrino in #906
- [CDF-22441] ๐งฑ Robustify reading feature flags by @doctrino in #908
- [CDF-22474] ๐ปโโ๏ธ Classic Resource Folder by @doctrino in #907
- [CDF-22341] ๐ชฆRip cognite modules by @ronpal in #896
- [CDF-22341] ๐๏ธ modules init --all (--clean) copies all files from _builtin_modules to org_dir/modules by @ronpal in #916
- [CDF-22496] ๐ธ Instrument feature flag by @doctrino in #915
- [CDF-22353] ๐ Module metrics by @ronpal in #917
- [CDF-22493] ๐ดIntroduce repo Command by @doctrino in #919
- [CDF-22495] ๐ฆญAdd back in skipped tests by @doctrino in #918
- [CDF-22492] ๐ทโโ๏ธ Fix creation config.dev.yaml by @doctrino in #920
- [CDF-22457] ๐คผ Migrations by @doctrino in #921
- [CDF-22507, CDF-22508, CDF-22509, CDF-22512]๐Last Minute Fixes by @doctrino in #922
- better user guide by @ronpal in #924
- Prepare 0.3.0a1 by @doctrino in #923
- fix: bump command by @doctrino in #926
- Prepare release 0.3.0a2 by @doctrino in #927
- ๐ผCI Remove extra jobs by @doctrino in #929
- friendlier message if no functions are found by @ronpal in #932
- Changed prompts that were confusing by @ronpal in #933
- Remove SDK Warnings by @doctrino in #934
- Fixing location filter by @ronpal in #936
- [CDF-22358] โถWorkflow trigger loader by @doctrino in #930
- [CDF-22467] ๐Version as string by @doctrino in #931
- [CDF-22525] ๐ฅบ Consistent use of org by @doctrino in #935
- Prepare release0.3.0a3 by @doctrino in #937
- ๐fix: pick up default org in cdm.toml by @doctrino in #939
- [CDF-22528] ๐ Support dict workflowtrigger by @doctrino in #940
- [CDF-22527] ๐งโ๐คโ๐งUserfriendly message if modules and config not found. by @doctrino in #941
- [CDF-22526] ๐คธโโ๏ธ Migration Fix by @doctrino in #942
- [CDF-22467] ๐ GraphQL Loader version bug by @doctrino in #943
- [CDF-22538] ๐ฆGroup without metadata by @doctrino in #946
- [CDF-22518] ๐ง GraphQL Parser by @doctrino in #944
- Prepare 0.3.0a4 by @doctrino in #948
- [CDF-22331, CDF-22544] ๐งExtraction Pipeline Config parse config as string by @doctrino in #950
- [CDF-22351] modules add subcommand by @ronpal in #955
- [CDF-21821, CDF-21860, CDF-22355] Split auth in inti and verify by @doctrino in #956
- [CDF-22414] ๐ฃ Hosted extractor source loader by @doctrino in #951
- [CDF-22345] ๐ฅฆ Support Sequences by @doctrino in #958
- [CDF-22564] ๐ Cleanup deploy/clean commands by @doctrino in #959
- [CDF-22579] ๐ฎโ๐จLocation filter with data models by @doctrino in #962
- Fix location filter dump by @doctrino in #964
- acl for sequences by @ronpal in #963
- Prepare 0.3.0a5 by @doctrino in #965
- [CDF-22563] ๐งโโ๏ธFix update by @doctrino in #970
- add bootcamp initial config for participants by @petreeb in #971
- [CDF-22565] ๐ซตBuild Selected Option by @doctrino in #961
- [CDF-22420, CDF-22424] adding plugins and updating feature_flags. Tracking use of plugins in MixPanel by @ronpal in #960
- Prepare 0.3.0a6 by @doctrino in #973
- [CDF-22632] ๐ฆฟTOML Error by @doctrino in #975
- [CDF-22635] ๐งโ๐ซLocation Testing by @doctrino in #977
- [CDF-22634] ๐ Migration from 0.2.x to 0.3 alpha by @doctrino in #976
- [CDF-22637] ๐ค Module upgrade verbose by @doctrino in #978
- Prepare 0.3.0a7 by @doctrino in #991
- ๐ค Tiny cleanup by @doctrino in #969
- [CDF-22641] ๐ฎโ๐จNo longer validate import of function in build step by @doctrino in #979
- [prep CDF-22639] ๐ State between build and depoy by @doctrino in #980
- [CDF-22415] ๐ Destination loader by @doctrino in #983
- [CDF-22416] ๐งโ๐ง Job loader by @doctrino in #984
- [CDF-22640, CDF-22639] ๐ข Deploy state by @doctrino in #981
- [CDF-22589] ๐Rework Pull Command by @doctrino in #985
- [CDF-22626] ๐ Rename Toolkit Group by @doctrino in #993
- [CDF-22540] ๐ Unify dump by @doctrino in #986
- [CDF-22590]๐ชฆ RIP Describe by @doctrino in #990
- [CDF-22346]๐ Support CogniteFile by @doctrino in #982
- [CDF-22656] ๐คจ Remove NodeAPICall option by @doctrino in #994
- [CDF-22655] ๐ Bootcamp special needs by @doctrino in #995
- [CDF-22352] ๐ถ Added deprecation warning by @doctrino in #996
- [CDF-22660] ๐โ
โ๏ธ Setup plugins by @doctrino in #998 - [CDF-22659] ๐ฌ Drop python3.9 support by @doctrino in #997
- [CDF-22657] ๐ Handle no .as_write by @doctrino in #999
- Automatic fix node by @doctrino in #1002
- Prepare 0.3.0b1 by @doctrino in #1001
- [CDF-22638] ๐คค Auto Release by @doctrino in #1000
- [CDF-22682] ๐ Bootcamp to test by @doctrino in #1004
- [CDF-22683] ๐งโ
โ๏ธ Auth Verify by @doctrino in #1005 - [CDF-22691] ๐Int version in Transformations by @doctrino in #1007
- [CDF-22636] ๐ดGraphQL Loader Fixes by @doctrino in #1006
- Prepare 0.3.0b2 by @doctrino in #1008
- ๐ Fix autorelase by @doctrino in #1010
- ๐Added back transformation loader tests by @doctrino in #1011
- [CDF-22636] ๐ฌ GraphQL Parser by @doctrino in #1012
- [CDF-22417] ๐บSupport hosted extractor mappings by @doctrino in #1014
- Prepare 0.3.0b3 by @doctrino in #1015
- Some branding artifacts by @ronpal in #1022
- fixed bug with performance and quality timeseries by @petreeb in #1026
- Adding locationfilter example to builtin modules by @ronpal in #1018
- Prepare 0.3.0b4 by @ronpal in #1029
- [CDF-22718] ๐ณ When running
deploy --dry-run, the Toolkit will not require write access to the CDF project. by @ronpal in #1031 - [CDF-22685] ๐ผ Builder Concept Part 1 by @doctrino in #1017
- [CDF-22685] ๐ผ Builder Concept Part 2 by @doctrino in #1032
- [CDF-22685] ๐ผ Builder Concept Part 3 by @doctrino in #1033
- Add authlib dependency and configure OAuth2 authentication by @gregertw in #846
- [CDF-22685] ๐ผ Builder Concept Part 4 by @doctrino in #1034
- [CDF-22745] ๐ฆ feat: Upload Cognite Files by @doctrino in #1036
- [CDF-22354] ๐ Docker image by @doctrino in #1035
- fix by @doctrino in #1043
- Prepare 0.3.0b5 by @doctrino in #1044
- Docker Push fix by @doctrino in #1048
- ๐ Stop trying to update docker files by @doctrino in #1057
- [CDF-22785] ๐ Expansion pattern for CogniteFile by @doctrino in #1058
- [CDF-22805, CDF-22806] ๐โ
โ๏ธ Core Extension Minimum and Full Models by @doctrino in #1061 - [CDF-22585] ๐งโ๐ง Repo init create GitHub workflow by @doctrino in #1047
- ๐ถSimplify core extension by @doctrino in #1064
- Module Dependencies and APM files for InRobot by @ronpal in #1060
- Prepare 0.3.0 by @doctrino in #1066
New Contributors
- @nodegard made their first contribution in #772
- @spex66 made their first contribution in #809
- @ana-gheorghe made their first contribution in #808
- @FrankDanielsenCognite made their first contribution in #842
- @petreeb made their first contribution in #971
Full Changelog: v0.2.20...v0.3.0