Code on external storage deleted in certain situations #24

Techy4198 · 2017-01-16T09:17:25Z

(As of 0.4.0) I really don't know what happened here but I'm lucky to have tested on an older copy of a sketch first. The compilation process seemed to go okay without throwing any errors, but when trying to launch the app it crashes (the code was fully working in 0.3.3, besides I do a lot of my own exception handling in it). When it crashed back to APDE I was greeted with no code at all. It's not scrolled off screen, does not return after fully restarting the app. I have also checked the sketchbook in a file explorer and that file is zero bytes in size. I consider this to be a critical bug, although I'm not sure if it triggers every time (and am not really willing to find out as I have no backups). If it is relevant, the app would have been locked to landscape orientation, so APDE would have been loading and changing orientation at the same time.

Note: I'm now using the account @1icri, so please mention/contact that account instead of Techy4198.

Techy4198 · 2017-01-16T09:19:52Z

Note: the app crash may well have been caused by outdated code, so I'm not so worried about that. The main issue to focus on is the code being deleted on compilation.

Calsign · 2017-01-16T15:35:38Z

I'm sorry that your code was deleted.

This is a definitely a critical bug. Thank you for reporting it through a medium other than Google Play with sufficient detail so that I can actually figure out what is causing the problem.

I am currently trying to reproduce the bug with the following steps:

New sketch with basic crashing code (below)
Lock orientation to landscape in sketch properties
Keep device orientation in portrait
Run sketch

Code:

void setup() {

}

void draw() {
  // Give enough time for the orientation change...
  if (millis() > 3000) {
    throw new RuntimeException();
  }
}

I have not been able to reproduce it on my phone - an Galaxy S6 running 6.0 - but am currently trying other devices and emulator configurations.

Can you possibly provide any more information about the bug? ~~Specifically, what version of Android do and what device do you have?~~ (It turns out that Google Play gives this information because you left a review.) Also, do you remember anything about the sketch crash? What exception did you receive, and was there anything else pertinent in the console?

I understand that you don't want to reproduce this bug with your own sketches, but it would be greatly appreciated if you could try to reproduce it on new sketches (as I have done above) and after backing up your sketchbook folder. It may be the case the bug only occurs randomly or under certain conditions beyond those described above, meaning that I need to do more steps to reproduce it.

Calsign · 2017-01-16T15:36:43Z

I am unable to reproduce the bug on my ASUS Nexus 7 (2012) running Android 6.0.

Calsign · 2017-01-16T15:50:02Z

I am unable to reproduce this bug in emulators running Android 6.0, 5.0, or 4.4...

I am beginning to think that there must be additional steps necessary to reproduce beyond the orientation changes and crash described above, especially considering that your device runs Android 6.0, which I have now tested three different times.

Edit: I should probably mention @Techy4198

Calsign · 2017-01-16T16:03:49Z

Here's another important question: Where do your store your sketches - on the internal storage or the external storage? You can see from Settings > General > Sketchbook Location.

Edit: I am unable to reproduce the bug on my S6 with the sketchbook in either the internal or external storage.

Techy4198 · 2017-01-16T16:14:43Z

There was no exception in the console, probably because of the app rotating (that seems to clear the console - is that another bug?), so I have no idea what caused the crash unfortunately. The code did involve shaders though, and I have a large portion of the same code in another sketch. Once that sketch is in a state where it can hopefully be compiled, I'll upload it here. Sketches are stored on external storage, but I didn't get the "can't access external storage" issue that others are getting.
I'll make backups and then get the code in a working state, should be able to upload the code within a few hours.

Calsign · 2017-01-16T16:32:03Z

Thank you.

It does sound as though the console output being hidden after rotation is a bug. I am going to try looking into this as it may be connected to code-deletion bug.

Techy4198 · 2017-01-16T16:51:29Z

Another minor bug I noticed that I doubt needs its own report, there is no longer a 'Save sketch' button in the menu.
Currently bodging the relevant code into a state that it actually compiles (I was doing a large restructure).

Techy4198 · 2017-01-16T16:55:43Z

I managed to catch the crash exception - it's just complaining about where I've been using size(). Interestingly though, now my phone is refusing to auto-rotate back to portrait after an app crash, making it difficult to know if this will reproduce it.

Calsign · 2017-01-16T16:59:46Z

The missing "Save Sketch" is intentional because the sketch saves automatically when APDE closes. However, it does seem that it could be useful in situations such as this...

Interestingly, another review for v0.4.0 mentioned something about the size() function. I will look into this.

Techy4198 · 2017-01-16T16:59:55Z

I have successfully reproduced it with the attached sketch on external storage and orientation set to landscape, although it took about six tries and the results were different each time.
Crash.zip
I'm now more available than I was earlier, so ask as many questions as you need.

Calsign · 2017-01-16T17:01:35Z

Thank you!

Calsign · 2017-01-16T17:09:58Z

Does the sketch crash for you? It is not crashing on my phone.

Techy4198 · 2017-01-16T17:13:44Z

Hmm, strange. Maybe something weird happened when I installed 0.4.0 over the top of 0.3.3. Technically it should crash though, because I'm using displayWidth and displayHeight variables in size(), and the "no variables in size()" thing came to APDE with this update right?

Calsign · 2017-01-16T17:21:30Z

The "no variables in size()" error message shouldn't happen under these circumstances. It is normally called by the preprocessor, meaning it should report an error before the sketch is run, and displayWidth and displayHeight should be allowable variables in size() anyway.

Perhaps there is an issue with the version of the Processing library. If there is such an issue with the version of the library, then that is likely a separate issue from the one causing the code to be deleted.

Are you able to replicate the bug with the sample crashing code that I had above? If you can then these are definitely two separate issues and I think it is necessary to focus on the code-deleting bug first. If you can't then something really fishy is going on...

Techy4198 · 2017-01-16T17:39:47Z

Tried on a fresh install of the app and still doing the same, here is the console output:

smooth() can only be used inside settings()
OpenGL error 1280 at bot beginDraw(): invalid enum
When not using the PDE, size() can only be used inside settings().
Remove the size() method from setup(), and add the following:
public void settings() {
  size(1920, 1080, processing.opengl.PGraphics2D);
}
java.lang.IllegalStateException: size() cannot be used here, see https://processing.org/reference/size_.html
  at processing.core.PApplet.insideSettings(Unknown Source)
  at processing.core.PApplet.size(Unknown Source)
  at processing.test.droidtracker.DroidTracker.setup(DroidTracker.java:300)
  at processing.core.PApplet.handleDraw(Unknown Source)
  at processing.opengl.PGLES$AndroidRenderer.onDrawFrame(Unknown Source)
  at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1590)
  at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1286)

Calsign · 2017-01-16T17:47:05Z

When not using the PDE

Interesting. I don't know why it thinks that, but it is most likely caused by the update to Android mode packaged with v0.4.0. That should be separate from the code-deletion bug though.

Are you able to reproduce the code-deletion bug with a stripped-down crashing sketch, such as the one that I included in my initial response to this thread?

Techy4198 · 2017-01-16T17:59:36Z

Yep, again took about five tries but it certainly happened.

Calsign · 2017-01-16T18:32:04Z

I thought that this might have something to do with the available memory so I tested it on an emulator with only 128MB of RAM but was still unable to reproduce.

If I can't reproduce this then I may have to take some shots in the dark and ask you to test out a custom APK to see if they resolve the bug or not.

Techy4198 · 2017-01-16T18:44:55Z

I'd certainly be willing to do that if necessary, though I won't be available much tomorrow.

Calsign · 2017-01-16T18:56:51Z

I was just able to replicate the code deletion. It happened randomly and not as the result of a sketch crashing. I'm going to investigate further.

Techy4198 · 2017-01-16T19:51:25Z

Bear in mind that, on my end at least, the rotation thing increases the chance a lot. I feel like the conditions involve the way the app loads its previous state when rotating and when another app that was in front closes. Possibly something strange happening when the screen rotates while in the middle of loading a file, causing some background exception somewhere which nulls out the file?

Techy4198 · 2017-01-16T19:51:54Z

Did not mean to close, please ignore.

Calsign · 2017-01-16T20:10:22Z

Yes, that is very nearly exactly what I think is happening. When APDE is no longer in the foreground, the currently open files are saved to temporary storage. Then when the app is re-opened the files are loaded from temporary storage. This instance state change also occurs when the screen orientation changes, meaning that this process can occur twice when switching from an app with a different orientation - exactly as you describe.

Problems arise when, for some reason, the state information isn't loaded correctly, resulting in empty files and all of the code being deleted. The code isn't actually deleted from disk, however, until you close APDE again, overwriting the existing files with the new empty ones.

This means that, theoretically, you should be able to access your soon-to-be-deleted sketch files from your computer with a USB cable if you keep APDE open after experiencing the bug. That is, assuming that the deletion has not already occurred the first time that the state was restored (i.e. before the screen rotated back).

An interesting consequence of this should be that not all of your tabs are actually deleted. APDE only stores the current tab and the two adjacent tabs in memory (one to the left and one to the right), and the rest of the tabs are unloaded. If you create four tabs, all containing code, and then reproduce the code deletion bug, then the one tab that was unloaded should still keep its code while the other tabs have their code deleted.

How do I know this? APDE has had similar code-deletion issues before, but I thought that they had all been resolved in v0.2.3. Obviously this is not the case and the Material design changes have brought the bug back to light in all of its ugliness.

Also, just because I roughly know the cause of the issue does not mean that I can solve it. I need to be able to replicate it consistently on my end in order to see if the changes that I am making have any effect.

s-mv · 2018-01-11T09:01:44Z

It happens on my phone when I use the PVector class objects.

kasamikona · 2018-01-11T09:39:33Z

@StinkyVoid this shouldn't change based on what classes are used, but if it somehow does then it might reveal something new about the bug. Are you able to reproduce it clearing tabs when using PVector and not clearing them when not using it?

s-mv · 2018-01-11T12:33:59Z

Yes. For sure. But it isn't about some code vanishing. First my app said it has stopped, then I downloaded APDE again from https://www.dropbox.com/s/aurd10i2rage28u/APDE-bugfix-2017-12-08.apk?dl=0 to find out that the code worked properly. BUT sadly, my app works properly only it back is pressed. Then it says that it has stopped unfortunately. Now I stopped using PVectors.

…

On Jan 11, 2018 3:09 PM, "1icri" ***@***.***> wrote: @StinkyVoid <https://github.com/stinkyvoid> this shouldn't change based on what classes are used, but if it somehow does then it might reveal something new about the bug. Are you able to reproduce it clearing tabs when using PVector and not clearing them when not using it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfDeEpVLxAKAOVCWhOctEipOj9TZCKhFks5tJdbWgaJpZM4LkU6g> .

Calsign · 2018-01-11T13:45:08Z

@StinkyVoid I am confused by what you are describing. Did you mean to post this on #32 instead of here?

s-mv · 2018-01-11T14:06:05Z

No. It happens on my new phone and isn't #32 -ish at all. I just wanted to help... Now the thing is that, on Back Key, the program shows *unfortunately, xmpl has stopped *when using PVectors.

…

On Jan 11, 2018 7:15 PM, "William Smith" ***@***.***> wrote: @StinkyVoid <https://github.com/stinkyvoid> I am confused by what you are describing. Did you mean to post this on #32 <#32> instead of here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfDeEiSzLjquOndGN45Eev3_AIUxzzNiks5tJhBlgaJpZM4LkU6g> .

Calsign · 2018-01-11T15:54:00Z

So my understanding is that you are experiencing the following problems:

Using PVector in the code causes the sketch to crash when pressing the back button
When the sketch crashes, the code is deleted

Do I understand you correctly?

I am aware of a bug that causes static mode sketches (without setup() or draw()) to crash after the back button is pressed, but this has nothing to do with PVector. Is it possible that this is causing the crash instead?

Can you provide a minimal code sample that still reproduces the PVector problem you are describing?

The sketch crashing shouldn't cause code to be deleted in the latest bugfix APK - if it is then something is going very wrong. Is this actually what you are experiencing?

Calsign · 2018-01-11T16:52:36Z

@StinkyVoid I forgot to @ mention you in my last comment.

WGG25 · 2018-01-12T16:49:39Z

Hi, i had the same deletion issue happen to me either yesterday while quitting, or today while starting up. I "quit" by pressing the home button and "started" by selecting the app on the recent apps screen. Last thing i tried before quitting was compiling the sketch, which failed (had to leave so i put the phone away).
Android version: 6.0.1, Phone: Samsung Galaxy S5, App version: 0.4.0
Sketch save location was internal storage (storage/emulated/0), although the app says Primary External (is it normal to have it be external?).
I had rooted the phone before but had to do a factory reset not so long ago (think i still have the rooted boot stuff)

Calsign · 2018-01-12T17:14:51Z

@WGG25 Please try the bugfix APK. This should fix most, and hopefully all, code deletion bugs:

https://www.dropbox.com/s/aurd10i2rage28u/APDE-bugfix-2017-12-08.apk?dl=0

Your storage location is labelled as "primary external" as opposed to the internal storage, which maps to APDE's private storage (in /data/data/). The Android system provides APDE with the list of internal and external storage drives. On the S5, the first external storage drive maps to the public partition of what you refer to as the internal storage.

s-mv · 2018-01-13T11:24:26Z

@william Smith I am currently working with the budfix version. I found another bug which can only be explained by screenshots.

…

On Jan 12, 2018 10:57 PM, "William Smith" ***@***.***> wrote: @WGG25 <https://github.com/wgg25> Please try the bugfix APK. This should fix most, and hopefully all, code deletion bugs: https://www.dropbox.com/s/aurd10i2rage28u/APDE-bugfix-2017-12-08.apk?dl=0 Your storage location is labelled as "primary external" as opposed to the internal storage, which maps to APDE's private storage (in /data/data/). The Android system provides APDE with the list of internal and external storage drives. On the S5, the first external storage drive maps to the public partition of what you refer to as the internal storage. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfDeErh16Y1qi76oZ-RYIuTYadC7LDfSks5tJ5M-gaJpZM4LkU6g> .

WGG25 · 2018-01-16T17:02:52Z

@Calsign Tried the version you linked, but it happened again. I was looking at references for processing so i switched apps, then i noticed a tab - that i haven't visited for a bit - was "gone" (contents deleted).

Calsign · 2018-01-18T22:11:15Z

@WGG25 Code being deleted in an unused tab sounds like a variation of the bug that I haven't seen before. The bug so far seems to deal with either all of the tabs or just the three tabs centered about the currently selected one. This might indicate that deletion of the current tab, but not that of other tabs, has been fixed, but it's a little early to speculate that far.

It would be very useful if you could provide me with the debug logs for your code deletion. If you go to Settings > General and enable "Code Deletion Debug Logs", the debug logs will get saved to a file in the sketchbook. You need to reproduce the deletion again in order to record it in the log. The logs will be added to the end of the file each time you run the app. The timestamp is saved with the logs so please note at approximately what time you are able to reproduce the bug so that I can narrow down what could potentially be a very large file.

You don't need to try to reproduce the bug but I would certainly be very grateful and I am unable to fix the bug without these debug logs.

Calsign · 2018-01-18T22:13:58Z

@StinkyVoid Do you have the screenshots for this bug? You can upload screenshots if you comment through the web page rather than through email.

If this bug only exists in the bugfix version then it is reasonable to discuss it here for now, but if it can also be reproduced in v0.4.0 (currently released on Google Play) then you should make a separate issue.

s-mv · 2018-01-19T10:59:33Z

Wait a bit... I will soon send the screenshots. Currently I am on a serious project on the normal version so I can't take any risk to install the APK again.

…

On Jan 19, 2018 3:44 AM, "William Smith" ***@***.***> wrote: @StinkyVoid <https://github.com/stinkyvoid> Do you have the screenshots for this bug? You can upload screenshots if you comment through the web page rather than through email. If this bug only exists in the bugfix version then it is reasonable to discuss it here for now, but if it can also be reproduced in v0.4.0 (currently released on Google Play) then you should make a separate issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfDeEpd4E5QNuvXKoC1oB6BpXvihOc3iks5tL8IogaJpZM4LkU6g> .

WGG25 · 2018-01-19T16:09:49Z

Yeah, it's most likely that triple-tab-deletion issue, i had three tabs open, was working on the rightmost one and the middle tab was deleted. I might try reproducing it when i have time.

EmmanuelPil · 2018-03-09T11:17:16Z

Hi.
I love this App and I defenitly would pay for it.
But this is happening all the time on my Samsung Tab E 9.6 with KitKat 4.4.4.
And it made me swearing. I use it every day, and the bug has nothing to do with craching code or more than 1 tab. It only happens after running. Now after writing two lines I copy it to another sketch to prevent smashing my device to the ground.

EmmanuelPil · 2018-03-18T17:48:52Z

Well, I think I have to apologize. I hadn't read all comments and saw right after writing my last comment that there was a fix. I work all day long with this app and I have to report that never again my code was deletted with the last fix. Also the console keeps its text after going from vertical to horizontal e viceverca.
I have an issue however that the code in tabs not always auto format.

kasamikona · 2018-03-25T23:20:34Z

I can also confirm that I've not seen any more deletions. I also can't reproduce the issue via the method in #28. Even if it turns out to not be fully fixed but just extremely rare now, I think it's still ready for release.

Calsign · 2018-03-26T02:29:26Z

@1icri You are right, I need to push these changes to the main release channel. I should be able to do this next weekend.

Calsign · 2018-04-03T01:42:52Z

@1icri @nking07049925 @Friedolin2000 @ShreerangV @EmmanuelPil @WGG25 I have pushed v0.4.1-pre1 to the APDE Preview Channel, with all of the fixes so far:

https://plus.google.com/communities/117703520591262292626

If you want to get this release, then please join the community above and click the link in the pinned comment. You could also wait until it gets rolled out to you, probably some time in the next week and a half, and then install from Google Play like normal.

In addition to the code deletion bugfixes in the last Dropbox APK I posted above, I have re-implemented undo/redo history (it used to get deleted when you closed the app due to the bugfixes) and merged in several improvements from the master branch.

If there are no major difficulties I will start rolling out the release to everyone this week.

Thank you all for your help in debugging this issue so far. I know that there are still some rare instances of the code deletion occurring, but there are now mechanisms in place to (hopefully) prevent loss of data when this happens and tools for logging information that will be useful for debugging the code deletion when it does occur.

Calsign · 2018-07-06T06:43:15Z

I haven't gotten any reports of code deletion since the release of v0.4.1, and this is available to all users, not just the preview channel. I am assuming that the issue is fixed, but if any issues re-emerge then please re-open this issue.

kasamikona · 2018-07-06T08:48:18Z

Good to hear.

EmmanuelPil · 2018-07-06T09:03:11Z

Confirming; working on daily basis did not have any issue with code deletion.

kasamikona · 2018-07-27T08:20:20Z

Is this happening again in a new version? I don't see any actual reference to this in #49 though it did accidentally reference a lot of issue numbers in the log.

Calsign added bug critical labels Jan 16, 2017

Calsign self-assigned this Jan 16, 2017

Techy4198 changed the title ~~It ate my code~~ Code on external storage deleted in certain situations Jan 16, 2017

Techy4198 closed this as completed Jan 16, 2017

Techy4198 reopened this Jan 16, 2017

kasamikona mentioned this issue Mar 25, 2018

"Show Sketch Folder" opens recent files instead #36

Open

Calsign mentioned this issue May 16, 2018

Sketch sometimes clears after turning the screen off and on #40

Closed

Calsign closed this as completed Jul 6, 2018

s0ko1ex mentioned this issue Jul 21, 2018

APDE preview #49

Closed

Calsign reopened this Jul 26, 2018

Calsign closed this as completed Jun 10, 2021

Code on external storage deleted in certain situations #24

Code on external storage deleted in certain situations #24

Comments

Techy4198 commented Jan 16, 2017 • edited Loading

Techy4198 commented Jan 16, 2017

Calsign commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017

Calsign commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017 • edited Loading

Techy4198 commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017

Techy4198 commented Jan 16, 2017

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017

Calsign commented Jan 16, 2017

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017 • edited Loading

Calsign commented Jan 16, 2017

Techy4198 commented Jan 16, 2017 • edited Loading

Techy4198 commented Jan 16, 2017

Calsign commented Jan 16, 2017 • edited Loading

s-mv commented Jan 11, 2018

kasamikona commented Jan 11, 2018

s-mv commented Jan 11, 2018 via email

Calsign commented Jan 11, 2018

s-mv commented Jan 11, 2018 via email

Calsign commented Jan 11, 2018

Calsign commented Jan 11, 2018

WGG25 commented Jan 12, 2018

Calsign commented Jan 12, 2018

s-mv commented Jan 13, 2018 via email

WGG25 commented Jan 16, 2018

Calsign commented Jan 18, 2018

Calsign commented Jan 18, 2018

s-mv commented Jan 19, 2018 via email

WGG25 commented Jan 19, 2018

EmmanuelPil commented Mar 9, 2018 • edited Loading

EmmanuelPil commented Mar 18, 2018

kasamikona commented Mar 25, 2018 • edited Loading

Calsign commented Mar 26, 2018

Calsign commented Apr 3, 2018

Calsign commented Jul 6, 2018

kasamikona commented Jul 6, 2018

EmmanuelPil commented Jul 6, 2018

kasamikona commented Jul 27, 2018

Techy4198 commented Jan 16, 2017 •

edited

Loading

Calsign commented Jan 16, 2017 •

edited

Loading

Calsign commented Jan 16, 2017 •

edited

Loading

Calsign commented Jan 16, 2017 •

edited

Loading

Techy4198 commented Jan 16, 2017 •

edited

Loading

Techy4198 commented Jan 16, 2017 •

edited

Loading

Techy4198 commented Jan 16, 2017 •

edited

Loading

Techy4198 commented Jan 16, 2017 •

edited

Loading

Techy4198 commented Jan 16, 2017 •

edited

Loading

Calsign commented Jan 16, 2017 •

edited

Loading

EmmanuelPil commented Mar 9, 2018 •

edited

Loading

kasamikona commented Mar 25, 2018 •

edited

Loading