Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ZIP does not support timestamps before 1980 #6

Open
clach04 opened this issue Jan 25, 2020 · 6 comments
Open

ZIP does not support timestamps before 1980 #6

clach04 opened this issue Jan 25, 2020 · 6 comments

Comments

@clach04
Copy link
Contributor

clach04 commented Jan 25, 2020 •
edited
Loading

Workaround

Starting a shell session in the container and using a linux file system avoids problems, see the notes in the readme for starting a session (ideally with a name, -name=pebbleDev for persistence):

docker run --rm -it -v ~/pebble-dev/:/pebble/ bboehmke/pebble-dev

Thanks for this Docker image - its the one I've had the most success with so far.

Platform information (which I'm not sure is relevant but included for completeness).

Windows 10 64-bit with Docker 64-bit:

C:\code\docker\pebble\bboehmke>docker --version
Docker version 19.03.5, build 633a0ea

C:\code\docker\pebble\bboehmke>ver

Microsoft Windows [Version 10.0.17134.1184]

My simple demo batch script:

set IMAGE=bboehmke/pebble-dev

mkdir build

REM Works :-) interactive shell
REM docker run --rm -it -v %CD%\build:/pebble/ bboehmke/pebble-dev


docker run --rm -it -v %CD%\build:/pebble %IMAGE% pebble --version
docker run --rm -it -v %CD%\build:/pebble %IMAGE% pebble new-project test
docker run --rm -it -v %CD%\build\test:/pebble %IMAGE% pebble clean
docker run --rm -it -v %CD%\build\test:/pebble %IMAGE% pebble build

The compile part of the build stage is successful, the bundling into PBW is not:

.
..
...
[55/64] aplite | memory_usage_report: build/aplite/pebble-app.elf build/aplite/app_resources.pbpack
-------------------------------------------------------
APLITE APP MEMORY USAGE
Total size of resources:        4092 bytes / 128KB
Total footprint in RAM:         846 bytes / 24KB
Free RAM available (heap):      23730 bytes
-------------------------------------------------------
[56/64] diorite | pebble-app.raw.bin: build/diorite/pebble-app.elf -> build/diorite/pebble-app.raw.bin
[57/64] chalk | pebble-app.raw.bin: build/chalk/pebble-app.elf -> build/chalk/pebble-app.raw.bin
[58/64] basalt | pebble-app.raw.bin: build/basalt/pebble-app.elf -> build/basalt/pebble-app.raw.bin
[59/64] aplite | pebble-app.raw.bin: build/aplite/pebble-app.elf -> build/aplite/pebble-app.raw.bin
[60/64] diorite | inject-metadata: build/diorite/pebble-app.raw.bin build/diorite/pebble-app.elf build/diorite/app_resources.pbpack -> build/diorite/pebble-app.bin
[61/64] chalk | inject-metadata: build/chalk/pebble-app.raw.bin build/chalk/pebble-app.elf build/chalk/app_resources.pbpack -> build/chalk/pebble-app.bin
[62/64] basalt | inject-metadata: build/basalt/pebble-app.raw.bin build/basalt/pebble-app.elf build/basalt/app_resources.pbpack -> build/basalt/pebble-app.bin
[63/64] aplite | inject-metadata: build/aplite/pebble-app.raw.bin build/aplite/pebble-app.elf build/aplite/app_resources.pbpack -> build/aplite/pebble-app.bin
[64/64] app_bundle:  -> build/pebble.pbw
Waf: Leaving directory `/pebble/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/usr/local/lib/python2.7/zipfile.py", line 1158, in write
    zinfo = ZipInfo(arcname, date_time)
  File "/usr/local/lib/python2.7/zipfile.py", line 328, in __init__
    raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980

Build failed.

Timestamps in host are today (2020-01-25). I've not dug into this any further. Logging what I have at this point.
@clach04
Copy link
Contributor Author

clach04 commented Jan 25, 2020

Hacking zipfile:

    if date_time[0] < 1980:
        raise ValueError('ZIP does not support timestamps before 1980 - %r' % (date_time,))

and dumping into my path

env PYTHONPATH=../ pebble build

Seeing:

Waf: Leaving directory `/pebble/test/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/pebble/zipfile.py", line 1158, in write
    zinfo = ZipInfo(arcname, date_time)
  File "/pebble/zipfile.py", line 328, in __init__
    raise ValueError('ZIP does not support timestamps before 1980 - %r' % (date_time,))
ValueError: ZIP does not support timestamps before 1980 - (1970, 1, 1, 0, 0, 0)

Build failed.

@clach04
Copy link
Contributor Author

clach04 commented Jan 25, 2020

I'm going to guess this is a weird docker thing:

    raise ValueError('ZIP does not support timestamps before 1980 - %r' % ((date_time, filename),))
ValueError: ZIP does not support timestamps before 1980 - ((1970, 1, 1, 0, 0, 0), 'diorite/pebble-app.bin')

Build failed.
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan  1  1970 build/diorite/pebble-app.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/
total 0
drwxrwxrwx 1 root root  4096 Jan 25 19:16 .
drwxrwxrwx 1 root root  4096 Jan 25 19:16 ..
-rwxr-xr-x 1 root root  4092 Jan 25 19:16 app_resources.pbpack
-rwxr-xr-x 1 root root   826 Jan 25 19:16 appinfo.auto.c
-rwxr-xr-x 1 root root  4260 Jan 25 19:16 appinfo.auto.c.30.o
-rwxr-xr-x 1 root root   840 Jan 25 19:16 pebble-app.bin
-rwxr-xr-x 1 root root 48650 Jan 25 19:16 pebble-app.elf
-rwxr-xr-x 1 root root   840 Jan 25 19:16 pebble-app.raw.bin
-rwxr-xr-x 1 root root   799 Jan 25 19:16 pebble_app.ld.auto
drwxrwxrwx 1 root root     0 Jan 25 19:16 src
-rwxr-xr-x 1 root root   110 Jan 25 19:16 system_resources.resball
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-a*
-rwxr-xr-x 1 root root   840 Jan 25 19:16 build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 48650 Jan 25 19:16 build/diorite/pebble-app.elf
-rwxr-xr-x 1 root root   840 Jan 25 19:16 build/diorite/pebble-app.raw.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan 25 19:16 build/diorite/pebble-app.bin
pebble@76c4fbaaf893:/pebble/test$ ls -al build/diorite/pebble-app.bin
-rwxr-xr-x 1 root root 840 Jan 25 19:16 build/diorite/pebble-app.bin

NOTE one of the ls -al shows jan 1s (no year shown) and then later Jan 25th!? These were entered immediately after each other. There is a virus check running in the background in my host OS (I have no permissions to disable).

I also saw odd issues with the .pyc file having a later date than a modied py file for the zipfile hacks I was making so resorted to removing the pyc file each time.

Current hack:

    if date_time[0] < 1980:
        date_time = (2020, 1, 1, 0, 0, 0)  # clach04 hack, maybe use now/today instead?
        #raise ValueError('ZIP does not support timestamps before 1980 - %r' % ((date_time, filename),))  # clach04 diag hack

This lead to one successful build. Later (and so far consistently) getting errors (note includes debug):

[64/64] app_bundle:  -> build/test.pbw
<type 'tuple'>
"((2020, 1, 25, 19, 20, 31), 'appinfo.json')"
len(filename), len(extra) '(12, 0)'
len(filename), len(extra) '(12, 0)'
<type 'tuple'>
"((1970, 1, 1, 0, 0, 0), 'diorite/pebble-app.bin')"
len(filename), len(extra) '(22, 0)'
Waf: Leaving directory `/pebble/test/build'
Build failed
Traceback (most recent call last):
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 123, in process
    ret=self.run()
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/Task.py", line 47, in run
    return m1(self)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/.waf-1.7.11-951087d39789950ed009f0c86ce75e7b/waflib/extras/process_bundle.py", line 42, in run
    mkbundle.make_watchapp_bundle(timestamp=self.generator.bld.env.TIMESTAMP,appinfo=self.generator.bld.path.get_bld().find_node('appinfo.json').abspath(),binaries=binaries,js=[js_file.abspath()for js_file in js_files],outfile=outfile)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 397, in make_watchapp_bundle
    bundle.write(outfile, verbose)
  File "/home/pebble/.pebble-sdk/SDKs/current/sdk-core/pebble/common/tools/mkbundle.py", line 286, in write
    z.write(bf, os.path.join(f.subfolder, os.path.basename(bf)))
  File "/pebble/zipfile.py", line 1202, in write
    self.fp.write(zinfo.FileHeader(zip64))
  File "/pebble/zipfile.py", line 399, in FileHeader
    len(filename), len(extra))
error: ushort format requires 0 <= number <= USHRT_MAX

Moving into /tmp (vm) rather than native host drive avoided all problems.

Any ideas on this with Windows?

@bboehmke
Copy link
Owner

I have never used this image on windows.
I also never really used docker on windows at all.

The only thing I can imagine is that the time of the docker VM is maybe not set correctly.
A simple date call gives maybe more information.
This should show the current time inside the VM and the container.

@bboehmke
Copy link
Owner

Otherwise the

Moving into /tmp (vm) rather than native host drive avoided all problems.

does not really match this theory.

@clach04 clach04 mentioned this issue Jan 26, 2020
Closed
@clach04
Copy link
Contributor Author

clach04 commented Jan 26, 2020

Yeah super odd - current theory is either something in Windows (Symantec) or some sync issue between host/container.

I documented the current (reliable) workaround in the original description. Having git in the container helps but I now need a reliable way to edit/build. I opened issued #7 to discuss possible options.

Again thanks for this, I played with a couple of pebble containers and all had some major problems, this really has been the smoothest option since moving away from a VM (previously I was using a 32-bit VM but my machine has HyperV enabled and Virtual Box won't host 32-bit images because of that).

@bboehmke
Copy link
Owner

It seems that there is a base issue in docker for windows.

I tried a simple touch -m src/c/test.c which caused a

touch: setting times of 'src/c/test.c': Software caused connection abort

followed by a complete destruction of the volume mount which requires a restart of the docker daemon.

I dont think we can fix this problem in the image. (At least I currently dont have an idea how)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@clach04 @bboehmke