v0.14.0: July 2019 Release

Release v0.14.0

New Features

• Web portal:

  • New job submission page for pai protocol v2 #3026
  • Update home page to support dedicated vc #2995

• Python SDK:

  • Sdk release v0.4.00 #3018

• New scheduler:

  • Dedicated vc support #2960

• PAI vscode extension:

  • Submit job v2 support #2913
  • Add schema and snippets for YAML text editor. #2978

• Team storage plugin:

  • New team-wise manage cli #2943

Improvements

• Web portal:

  • Refine job detail page's task list #2953
  • add new webHDFSUri in env.js.template (#3048)
  • Tweak job submission page layout (#3043)
  • css tweak (#3041)
  • refine submit job UI page (#3037)
  • refine home page's error handling (#3196)
  • renew docker image list and add tooltip (#3181)
  • remove prettier config file (#3184)
  • remove tachyons css to avoid classname error (#3173)
  • Add confirm dialog before batch edit admin's password (#3174)
  • fix UI broken if choose all of the VC when create user (#3177)
  • redesign batch edit behavior (#3172)
  • trim the docker url after job submission; fix job detail page's clone button's padding (#3169)
  • Change 'Import CSV' to 'Create Bulk Users' in user management (#3136)
  • update command section's placeholder (#3150)
  • move documents link to top nav bar (#3126)
  • tweak home page's gpu chart's height (#3131)
  • display red border when a task role is invalid (#3072)
  • change label of container size to resources per instance (#3101)
  • add error message when command is empty (#3098)
  • disable edit user form's auto fill (#3107)

• Rest server:

  • Add get job api v2 #2851
  • Fix error in Docker runtime detection #2888
  • update user's vc permission when create/remove vc #2939
  • Remove deprecated api in rest server #2954
  • AAD & New Schema of User and Group (#3034)
  • change get vc api (#3149)

• Yarn cluster && Framework launcher:

  • Upgrade zookeeper version to 3.4.14 #2884
  • Upgrade to Zookeeper 3.4.14 #2911

• Deployement:

  • client tolerate datanode replacement failure #2895
  • "paictl service delete" supports to delete log #2926
  • Update maximum resources in yarn-site.xml #2942
  • 418.56 support #3020
  • upgrade pai version in k8s (609680)
  • upgrade pai version (#3056)
  • Remove duplicate version prefix (#3124)

• Plugins:

  • Upgrade fstream version to fix vulnerability #2886
  • Refine error message for marketplace #2998

• Security:

  • Update js-yaml #2890 #2891 #2892
  • Add dependabot config file #2893
  • Update handlebars #2899 #2900
  • Update dependabot config file #2914
  • Update is-my-json-valid #2931
  • Update twisted #2912

Documentation

• Rewording and some format fixes. #2927
• Chinese translation and placeholder. #2919
• add submit v2 job to readme #3017)
• api doc update (#3216)
• add release note (#3204)
• pai upgrade doc (#3195)
• add job submission docs (#3183)
• change link of external project like python sdk (#3237)

Bug Fixes

• Web portal:

  • hide 0 gpu nodes from Available Nodes Chart #2915
  • fix job detail page's gpu attributes display bug. #3027
  • disable submit if command is empty (#3080)
  • auto remove empty lines in command (#3074)
  • keep mount config selection state （0af835)
  • fix the set-state warning after clone job (#3070)
  • fix clone job bug (#3068)
  • fix <p> tag and prop-types warning (#3067)
  • Return empty command if no teamwise mounts (4afd2e)
  • Adjust team-mount-list view (1b4190)
  • Align job submit page's submission section to task role (#3065)
  • Add tooltips to job submission page's field label. (#3046)
  • remove plugin in webportal config (#3063)
  • remove pylon address dependency (#3040)
  • fix export yaml bug (#3047)
  • fix webhdfs wrong request (#3044)
  • customized docker image inputField may disappear (#3033)
  • add dependency of joi for node server (#3031)
  • remove duplicated v of feedback (#3230)
  • change webportal doc link (#3229)
  • fix stdout/stderr's full log link bug when pylon is not used (#3219)
  • change tutorial link of home page (#3213)
  • fix bug of GPU available number (#3210)
  • hot fix for hdfs CORS problem (#3145)
  • fix docker bug #3134
  • hot fix hdfscli proxy problem (#3130)
  • fix virtual cluster's default value after job clone (#3128)
  • refine hdfs check for robustness (#3116)
  • change to lowercase letter for 'Completion Policy' (#3119)
  • fix data command error (#3111)
  • fix job submission page's jobRetryCount and taskRetryCount field (#3112)
  • redict v2 job to default submission papge if plugin not installed (#3091)
  • fix docker (#3097)
  • add empty key check to key-value list control (#3096)
  • change command section's default comments to placeholder (#3095)
  • disable submit if command is empty (#3080)
  • fix deployment field missing bug (#3238)
  • [Web Portal] fix port list bug (#3240)

• Rest server:

  • Add quotes for masked secrets field in protocol
  • Trap SIGTERM in entrypoint to avoid yarn container early stop #2947
  • fix bug #3009
  • Update http errors in get job v2 #3022
  • User Migrate Script Fix. (#3090)
  • Fix issue in updateUserVirtualCluster of rest-server (565073)
  • Fix user migration issue (#3036)
  • api permission fix (#3211)
  • Fix AAD group in dedicated vc create/delete (#3143)
  • API to create vc and remove vc and do the same operation to group (#3064)
  • Groupname schema (#3099)

• Hadoop:

  • Increase HDFS client default replica to 3 #2924
  • port conflict #3012
  • Pre check vclist from yarn and remove the vc type group which is not in. (#3158)
  • fix and improvement (#3093)

• Deployment:

  • fix configMigration srcipt bug(#3220)
  • user migrate script issue fix (#3209)
  • Move npm version to dockerfile #3109 (#3110)

Known Issues

• All lines in command will be concatenated by &&, so use # or \ in the command will cause bugs. This will be fixed in the future.
• Based on official doc, the different gpu driver versions may support different cuda versions. As our tests, current 384.111 gpu driver version does not support cuda10 image.