I have lost all my jobs / runs / trials since last week

[arminvburren]

Hello,

I am not sure if this related to a recent update but as of last week there was an update in the UI and now I can't find the tab with "Experiments / trial names / jobs". There's only an "Experiment" tab and it says "no Jobs" even though there was a lot of jobs before.

Also when i try to read the doc there is nothing that looks like what I was using before. Here are images of the script I was using to launch my jobs.

Can anyone explain what is happening ?

[qidewenwhen]

Hi @arminvburren, thanks for reaching out!

FYI we recently updated the SageMaker Experiment SDK, which is now part of the SageMaker SDK and provides easier way to create experiment resources. And we updated the UI as well in order to simplify the data scientist work.
Please find more information in:

• Blog: https://aws.amazon.com/blogs/machine-learning/next-generation-amazon-sagemaker-experiments-organize-track-and-compare-your-machine-learning-trainings-at-scale/
• Developer Guide: https://docs.aws.amazon.com/sagemaker/latest/dg/experiments-create.html

We recommend to use the new Run interface documented in resources above so that you can get benefits of the new experiment UI and SDK.
For how to retrieve your previous job data, please look at jkasiraj@'s comment below.

[jkasiraj]

Hi @arminvburren, one of the changes that was made with the new UI was to hide auto created jobs in the Experiments view by default. If you click the settings icon that currently says 5/6 and update it to select all fields, you should be able to see your jobs.

Do you see your data after doing so? I will also bring up this feedback to the team, as there has been discussion over whether or not to show them by default

[arminvburren]

Yes I see them now thanks. Can you explain to me in like a sentence what's exactly the difference between a Run and a job ? Also, what has become of trials ?

If jobs are 'current training processes' and Run are 'finished processes' i do think you should absolutely show them since I was also confused by that (I was launching trainings and they were not appearing nowhere)

[arminvburren]

By the way I'd also like to be able to check if a job is running and stop it, but i can't see exactly where this is

[jkasiraj]

Yes I see them now thanks. 

Glad it's working now!

Can you explain to me in like a sentence what's exactly the difference between a Run and a job ?

The sentence version:

• A job maps to a Trial Component and a Trial Component is a Run.

The slightly longer version:

• SageMaker jobs are run on containers for training, processing, tuning, transforming
• Trial Components are what existed in the old UI and continue to be carried forward in the new UI
• Job Trial Components are automatically created when a job is created in order to keep track of useful data
• Runs are Trial Components and so a Job Trial Component is a type of Run. Conceptually, a run consists of all the inputs, parameters, configurations, and results for one iteration of model training (Ref).

Also, what has become of trials ?

Trials are now Run Groups, which is a column in the Runs table

If jobs are 'current training processes' and Run are 'finished processes' 

This is incorrect, a run can be finished or in progress and a job can too

i do think you should absolutely show them since I was also confused by that (I was launching trainings and they were not appearing nowhere

Will pass it on!

By the way I'd also like to be able to check if a job is running and stop it, but i can't see exactly where this is

Makes sense, we are definitely thinking about this as a future improvement. In the meantime you could use the SageMaker console if you have access to it in the AWS console

[jkasiraj]

Marking this issue as closed, an update to allow jobs to be visible by default will be coming out in the near future