ask about the results

[wentaoxandry]

Hi, @georgesterpu,

Sorry I have a question about the program. I have trained the models for many epochs, but until now (20 epochs), the decoded results are all the same, I want to ask if it is normal at the beginning the results are the same?

Here are some results at 20 epochs:
Test/6378096013485054233/00006 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THAT'S WHY I'M LOSING MY VOICE]
Test/6355556454612448247/00010 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [AND IN THE PROCESS OF ME NOT OFFENDING GOD]
Test/6374083655167624227/00107 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [DEPENDED ON TRADE]
Test/6344709944203120665/00030 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THAT'S A SHOCKING PERCENTAGE]
Test/6362913733590438579/00099 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WHEN I GET HOME]
Test/6347195441777342774/00010 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [OUR CONTINUED PRESSURE]
Test/6343252661930009508/00117 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [ACCORDING TO LEGEND]
Test/6339758276407584924/00030 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WHICH IS ONLY 10 MINUTES AWAY]
Test/6381853250875656673/00019 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [TODAY ALMOST 50]
Test/6334563083966338309/00024 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [IF IT WAS GOING TO BE INCREASED]
Test/6361814651459450364/00025 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THE PRIME MINISTER SLAPS HIM DOWN]
Test/6349793037997935601/00128 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [ESPECIALLY AS IT RELATES TO CHILDREN]
Test/6369259547770281088/00021 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BUT SLIGHTLY REBELLIOUS SOCKS]
Test/6357006006074844331/00013 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [AFTER ALL I'VE BEEN THROUGH]
Test/6377794506780964609/00004 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [I'M SORRY THAT DOESN'T MEET WITH YOUR APPROVAL]
Test/6357006006074844331/00003 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BUT IT'S NOT ALL BAD NEWS]
Test/6392896900283869820/00003 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [IT MAY TAKE SOME TIME]
Test/6334934169140712754/00001 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BUT WHAT A SURPRISE WHEN YOU COME IN]
Test/6354988230439182363/00085 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [MY FIRST GRANDDAUGHTER]
Test/6388126909604862142/00013 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [ANYTHING CAN HAPPEN]
Test/6392197250111351363/00024 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [EVERYONE HAS GONE HOME HAPPY AND THAT'S WHAT IT'S ALL ABOUT]
Test/6362913733590438579/00082 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [SPACE WAS STILL THE FINAL FRONTIER]
Test/6386959537493875163/00014 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WE NEED TO BE A TEAM]
Test/6384829663342218937/00013 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [FROM TIME TO TIME]
Test/6361725745636423155/00006 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BUT SHE'S NOT CONFUSED]
Test/6341242617105082637/00032 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [IF YOU CAN'T GO UP]
Test/6350523611934991188/00027 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THIS IS ONE OF THE QUESTIONS]
Test/6370728426585516325/00014 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [I THINK YOU DESERVE]
Test/6392568335285725796/00003 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THE WAY THE MARKET IS]
Test/6381766922032944158/00007 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [VERY HEAVILY TAXED]
Test/6373112133434795034/00005 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [FOR SOMETHING COMPLETELY DIFFERENT]
Test/6368494184598133861/00041 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [IT'S DOWN TO OUR BRAIN GETTING THINGS WRONG]
Test/6340299442417279404/00006 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [SO RATHER THAN JUST RELYING ON THIS INFORMATION]
Test/6369912812296002056/00009 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BACK TO THE HISTORY BOOKS]
Test/6351265782283739990/00032 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [TRAPPED UNDER HEAVY ARTILLERY FIRE]
Test/6384361941273252259/00012 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [YOU'RE AN IDIOT]
Test/6347195441777342774/00139 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [IN ALL SERIOUSNESS]
Test/6343252661930009508/00020 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [FOR THE FIRST TIME]
Test/6380036479839881698/00012 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WITH OTHER PEOPLE]
Test/6360322579951237696/00006 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WHAT WE'VE GOT HERE IS AN OPPORTUNITY TO UNDERSTAND ONE OF THESE MONUMENTS IN A MODERN]
Test/6380021017957616097/00021 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [COME TO MY ROOM]
Test/6351810813633601326/00016 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WHY DON'T WE ALL JUST GET ALONG]
Test/6349847154585871175/00014 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [HERSELF CURRENTLY LEARNING TO SIGN]
Test/6368494184598133861/00030 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [REPRESENTED BY THIS BUNCH OF BANANAS]
Test/6374454740211592999/00010 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [MUCH MORE DRIVEN]
Test/6375923619026758902/00020 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [AS LONG AS THEY SEE THE SPARK OF SUCCESS]
Test/6353909764151156687/00001 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [FURNITURE OR OBJECTS LIKE THAT]
Test/6335305254315087182/00014 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [BUT WE'RE GOING TO IMPROVE ON THINGS LIKE THE GREEN ROOF]
Test/6373112133434795034/00002 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [FLOWERS AND EVEN VEGETABLES]
Test/6374060462213818017/00008 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [AND MORE MONEY FOR FIFA]
Test/6385475196796375542/00001 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [I WOULD LIKE TO HELP]
Test/6367373198133943465/00022 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [YOU GO TO BED WITH IT]
Test/6369599709180191149/00011 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [I'M NOT SURE WHAT'S THE MOST DIFFICULT THING TO GET RIGHT]
Test/6355788382846429521/00005 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [TWO APPOINTMENTS LATER]
Test/6348038114360790454/00004 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [WITHOUT FURTHER ADO]
Test/6364791063795580193/00006 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [THE PERFECT FAMILY]
Test/6362913733590438579/00185 THE THE THE THE THE THE THE THE THE THE THE THE THE THE THE [YOU CAN'T REALLY SAY]

[wentaoxandry]

@georgesterpu , sry, have this problem solved.

[georgesterpu]

Hi @wentaoxandry
Apologies for replying so late.
Would you mind explaining how you were able to solve this problem ?
Other people may have the same question, and they may find your answer useful.

[wentaoxandry]

Hi @georgesterpu,
I made a mistake, I have changed video_processing to None, Then I changed that to feature, it is ok.

At the beginning, the results changed, here is the results from the first 5 epochs
Test/6353179190214044575/00010 THET THANG [IT'S ALL BASED ON THIS 58 PAGE REPORT PREPARED FOR THE EUROPEAN COMMISSION]
Test/6379224730890468911/00025 THENG [GET DOWN TO YOUR LOCAL ONE]
Test/6338130913429528164/00028 THENG [VISITING ALMOST ALL PARTS OF THE EMPIRE]
Test/6362096830810797583/00021 THENG [IT'S A TERRIBLE IDEA]
Test/6350222105230808621/00007 THENG [I'M WITH MARK TWAIN LIES]
Test/6359580409472005362/00001 THENG [THEY HAVE NO IDEA WHAT TO EXPECT]
Test/6384829663342218937/00039 THENG [REGARDLESS OF WHAT HAPPENS]
Test/6335676339489461610/00014 THENG [WE DON'T WANT TO SPEND MORE THAN]
Test/6330311066473698535/00011 THENG [AND FOR ME THE SURPRISE WAS]
Test/6382618614178237455/00017 THET THANG [TO DIVIDE THE ROOM INTO TWO PARTS]
Test/6350921755403330389/00056 THENG [GIVING HOPE AND CARE TO OTHERS IS SOMETHING OUR ARMED FORCES DO WITHOUT QUESTION]

But now, after 20 epochs it all the same again:
Test/6353179190214044575/00010 THING [IT'S ALL BASED ON THIS 58 PAGE REPORT PREPARED FOR THE EUROPEAN COMMISSION]
Test/6379224730890468911/00025 THING [GET DOWN TO YOUR LOCAL ONE]
Test/6338130913429528164/00028 THING [VISITING ALMOST ALL PARTS OF THE EMPIRE]
Test/6362096830810797583/00021 THING [IT'S A TERRIBLE IDEA]
Test/6350222105230808621/00007 THING [I'M WITH MARK TWAIN LIES]
Test/6359580409472005362/00001 THING [THEY HAVE NO IDEA WHAT TO EXPECT]
Test/6384829663342218937/00039 THING [REGARDLESS OF WHAT HAPPENS]
Test/6335676339489461610/00014 THING [WE DON'T WANT TO SPEND MORE THAN]
Test/6330311066473698535/00011 THING [AND FOR ME THE SURPRISE WAS]
Test/6382618614178237455/00017 THING [TO DIVIDE THE ROOM INTO TWO PARTS]
Test/6350921755403330389/00056 THING [GIVING HOPE AND CARE TO OTHERS IS SOMETHING OUR ARMED FORCES DO WITHOUT QUESTION]

Here are some logs:
Average batch_loss as epoch 1 is 3.4890882644100465
--- new best score ---

Average batch_loss as epoch 2 is 4.203346429478308
Average batch_loss as epoch 3 is 3.5442280127395014
Average batch_loss as epoch 4 is 2.9710363551943444
--- new best score ---

Average batch_loss as epoch 5 is 2.7390092682147373
--- new best score ---

character: 86.2528% word: 100.0000%
Average batch_loss as epoch 6 is 2.6276948267381877
--- new best score ---

Average batch_loss as epoch 7 is 2.629956953446564
Average batch_loss as epoch 8 is 2.664978136976807
Average batch_loss as epoch 9 is 2.7064000020609633
Average batch_loss as epoch 10 is 2.6471189719302806
character: 77.7236% word: 95.1781%
Average batch_loss as epoch 11 is 2.6187857994875308
--- new best score ---

Average batch_loss as epoch 12 is 2.631853339662216
Average batch_loss as epoch 13 is 2.65958504972991
Average batch_loss as epoch 14 is 2.6713306309767146
Average batch_loss as epoch 15 is 2.828360452390359
character: 79.9721% word: 99.0378%
Average batch_loss as epoch 16 is 2.6376371674409316
reduce learning rate to: 0.085
Average batch_loss as epoch 17 is 2.6348431855748653
--- new best score ---

Average batch_loss as epoch 18 is 3.4625331254972926
Average batch_loss as epoch 19 is 3.207513002875429
Average batch_loss as epoch 20 is 2.879233657844812
character: 88.1714% word: 99.9441%
Average batch_loss as epoch 21 is 3.2664754792276622
Average batch_loss as epoch 22 is 3.2363471412757416
reduce learning rate to: 0.07225000000000001
Average batch_loss as epoch 23 is 2.963414747152269
--- new best score ---

I used learning rate as 0.1, and if the results not getting better in 5 epochs, the learning rate will reduce 25%. I want to ask if I should just wait until better loss error, then the results will be fine? Or I make some mistake again?

Thank you so much

[javadpeyman]

same problem...
@wentaoxandry did you solve the problem?
@georgesterpu any idea?

[georgesterpu]

Not really sure, but the learning rate seems to be too high.
In this work we used a learning rate of 0.001

[wentaoxandry]

Hi @javadpeyman @georgesterpu,

I have solved the problem, for me the reasons for that can be:

1. video_processing set to None, after I change that, the results also variant.
2. maybe you have wrong audio or video features.
   I run now learning rate with 0.001 and it works now
   @georgesterpu thank you for your contribution and help.

[javadpeyman]

@wentaoxandry thank you,
did you run with audio? I want to just use video, without audio?

[wentaoxandry]

@javadpeyman I run with both audio and video features. Maybe there is something wrong with your video features?

[javadpeyman]

@wentaoxandry thank you so much. I will check that.