Skip to content

Buildfixes: Better linux packaging and https #5438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from
Closed

Buildfixes: Better linux packaging and https #5438

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5de6c1f
Add option to disable doc packaging
NicoHood Oct 3, 2016
b637e38
Fix light_bundle package option
NicoHood Oct 3, 2016
4cee989
Added option local_sources for better linux packaging with predownloa…
NicoHood Oct 3, 2016
f224a39
Use https in build downloads
NicoHood Oct 3, 2016
58b8b3f
Merge remote-tracking branch 'upstream/master' into buildfix
NicoHood Nov 26, 2016
11e8091
Merge remote-tracking branch 'upstream/master' into buildfix
NicoHood Dec 23, 2016
f29abf8
Added build option for disabling arduino-builder
NicoHood Dec 23, 2016
4f41384
Added platform.txt files back to hardware folder. Fixes arduino-builder
NicoHood Dec 23, 2016
054bc50
Merge remote-tracking branch 'upstream/master' into buildfix
NicoHood Dec 27, 2016
35e5f33
Revert "Added platform.txt files back to hardware folder. Fixes ardui…
NicoHood Dec 27, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 32 additions & 62 deletions build/build.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,8 @@
</target>

<!-- copy library folder -->
<target name="assemble-libraries" unless="light_bundle">
<target name="assemble-libraries" depends="assemble-libraries-full, assemble-libraries-light" />
<target name="assemble-libraries-full" unless="light_bundle">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guees this is to keep the empty libraries folder. Is it really needed?

<copy todir="${target.path}/libraries">
<fileset dir="../libraries"/>
</copy>
Expand All @@ -239,6 +240,9 @@
<download-library name="SD" version="1.1.0"/>
<download-library githubuser="Adafruit" name="Adafruit_CircuitPlayground" version="1.6.4"/>
</target>
<target name="assemble-libraries-light" if="light_bundle">
<mkdir dir="${target.path}/libraries" />
</target>

<macrodef name="download-library">
<attribute name="name"/>
Expand All @@ -257,13 +261,17 @@
</macrodef>

<!-- copy hardware folder -->
<target name="assemble-hardware" unless="light_bundle">
<target name="assemble-hardware" depends="assemble-hardware-full, assemble-hardware-light" />
<target name="assemble-hardware-full" unless="light_bundle">
<copy todir="${target.path}/hardware">
<fileset dir="../hardware">
<exclude name="arduino/sam/**"/>
</fileset>
</copy>
</target>
<target name="assemble-hardware-light" if="light_bundle">
<copy file="../hardware/package_index_bundled.json" todir="${target.path}/hardware" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case the file ../hardware/package_index_bundled.json is wrong, it should be a json with zero platforms defined. I don't know maybe the best thing to do is to add a check to the IDE...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a lot of problems with this file. Currently I add a copy fo the avr core to the hardware path with a different name and patches to use the archlinux tools. However If i define this package in this json the IDE always crashs on a name not equal to "arduino". I am not sure if the file is required in this case.

If it should be empty, maybe you should add a stub with no data in it instead. However this works for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package_index_bundled.json is, basically, a core-package json that describe what's bundled inside the IDE hardware folder. The IDE merges the content of package_index_bundled.json with the general package_index.json that is downloaded from arduino.cc. The bundled index has priority over the downloaded index.

If you want to make a real no-bundle (or light) version of the IDE, the correct thing to do is to create an empty (== with zero platforms defined) package_index_bundled.json.

</target>

<!-- - - - - - - - - -->
<!-- Revision check -->
Expand Down Expand Up @@ -472,21 +480,7 @@
<copy file="macosx/liblistSerials-${LIBLISTSERIAL-VERSION}/osx/liblistSerialsj.dylib" todir="macosx/work/${staging_hardware_folder}/../lib/" />
<chmod perm="755" file="macosx/work/${staging_hardware_folder}/../lib/liblistSerialsj.dylib" />

<delete dir="${staging_folder}/arduino-builder-macosx" includeemptydirs="true"/>
<mkdir dir="${staging_folder}/arduino-builder-macosx"/>
<antcall target="untar">
<param name="archive_file" value="./arduino-builder-macosx-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="archive_url" value="https://downloads.arduino.cc/tools/arduino-builder-macosx-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="final_folder" value="${staging_folder}/arduino-builder-macosx/arduino-builder" />
<param name="dest_folder" value="${staging_folder}/arduino-builder-macosx" />
</antcall>
<copy file="${staging_folder}/arduino-builder-macosx/arduino-builder" tofile="macosx/work/${staging_hardware_folder}/../arduino-builder" />
<chmod perm="755" file="macosx/work/${staging_hardware_folder}/../arduino-builder" />
<move file="${staging_folder}/arduino-builder-macosx/tools" tofile="macosx/work/${staging_hardware_folder}/../tools-builder"/>
<copy todir="macosx/work/${staging_hardware_folder}" overwrite="true">
<fileset dir="${staging_folder}/arduino-builder-macosx/hardware" includes="*.txt"/>
</copy>
<delete dir="${staging_folder}/arduino-builder-macosx" includeemptydirs="true"/>
<antcall target="build-arduino-builder" />

<antcall target="portable-${portable}">
<param name="parentdir" value="macosx/work/${staging_hardware_folder}/.." />
Expand Down Expand Up @@ -706,21 +700,7 @@
<param name="JVM" value="${LINUXARM_BUNDLED_JVM}"/>
</antcall>

<delete dir="${staging_folder}/arduino-builder-arm" includeemptydirs="true"/>
<mkdir dir="${staging_folder}/arduino-builder-arm"/>
<antcall target="untar">
<param name="archive_file" value="./arduino-builder-arm-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="archive_url" value="https://downloads.arduino.cc/tools/arduino-builder-arm-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="final_folder" value="${staging_folder}/arduino-builder-arm/arduino-builder" />
<param name="dest_folder" value="${staging_folder}/arduino-builder-arm" />
</antcall>
<copy file="${staging_folder}/arduino-builder-arm/arduino-builder" tofile="linux/work/arduino-builder" />
<chmod perm="755" file="linux/work/arduino-builder" />
<move file="${staging_folder}/arduino-builder-arm/tools" tofile="linux/work/tools-builder"/>
<copy todir="linux/work/hardware" overwrite="true">
<fileset dir="${staging_folder}/arduino-builder-arm/hardware" includes="*.txt"/>
</copy>
<delete dir="${staging_folder}/arduino-builder-arm" includeemptydirs="true"/>
<antcall target="build-arduino-builder" />

<antcall target="avr-toolchain-bundle">
<param name="unpack_target" value="untar"/>
Expand All @@ -738,21 +718,7 @@
<param name="JVM" value="${LINUX32_BUNDLED_JVM}"/>
</antcall>

<delete dir="${staging_folder}/arduino-builder-linux32" includeemptydirs="true"/>
<mkdir dir="${staging_folder}/arduino-builder-linux32"/>
<antcall target="untar">
<param name="archive_file" value="./arduino-builder-linux32-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="archive_url" value="https://downloads.arduino.cc/tools/arduino-builder-linux32-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="final_folder" value="${staging_folder}/arduino-builder-linux32/arduino-builder" />
<param name="dest_folder" value="${staging_folder}/arduino-builder-linux32" />
</antcall>
<copy file="${staging_folder}/arduino-builder-linux32/arduino-builder" tofile="linux/work/arduino-builder" />
<chmod perm="755" file="linux/work/arduino-builder" />
<move file="${staging_folder}/arduino-builder-linux32/tools" tofile="linux/work/tools-builder"/>
<copy todir="linux/work/hardware" overwrite="true">
<fileset dir="${staging_folder}/arduino-builder-linux32/hardware" includes="*.txt"/>
</copy>
<delete dir="${staging_folder}/arduino-builder-linux32" includeemptydirs="true"/>
<antcall target="build-arduino-builder" />

<antcall target="avr-toolchain-bundle">
<param name="unpack_target" value="untar"/>
Expand All @@ -770,21 +736,7 @@
<param name="JVM" value="${LINUX64_BUNDLED_JVM}"/>
</antcall>

<delete dir="${staging_folder}/arduino-builder-linux64" includeemptydirs="true"/>
<mkdir dir="${staging_folder}/arduino-builder-linux64"/>
<antcall target="untar">
<param name="archive_file" value="./arduino-builder-linux64-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="archive_url" value="https://downloads.arduino.cc/tools/arduino-builder-linux64-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="final_folder" value="${staging_folder}/arduino-builder-linux64/arduino-builder" />
<param name="dest_folder" value="${staging_folder}/arduino-builder-linux64" />
</antcall>
<copy file="${staging_folder}/arduino-builder-linux64/arduino-builder" tofile="linux/work/arduino-builder" />
<chmod perm="755" file="linux/work/arduino-builder" />
<move file="${staging_folder}/arduino-builder-linux64/tools" tofile="linux/work/tools-builder"/>
<copy todir="linux/work/hardware" overwrite="true">
<fileset dir="${staging_folder}/arduino-builder-linux64/hardware" includes="*.txt"/>
</copy>
<delete dir="${staging_folder}/arduino-builder-linux64" includeemptydirs="true"/>
<antcall target="build-arduino-builder" />

<antcall target="avr-toolchain-bundle">
<param name="unpack_target" value="untar"/>
Expand Down Expand Up @@ -827,6 +779,24 @@
<exec executable="./linux/work/arduino" spawn="false" failonerror="true"/>
</target>

<target name="build-arduino-builder" unless="no_arduino_builder">
<delete dir="${staging_folder}/arduino-builder-${platform}" includeemptydirs="true"/>
<mkdir dir="${staging_folder}/arduino-builder-${platform}"/>
<antcall target="untar">
<param name="archive_file" value="./arduino-builder-${platform}-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For linuxarm the filename is arduino-builder-linuxarm-1.3.23.tar.bz2, but the file is actually called arduino-builder-arm-1.3.23.tar.bz2. I'm going to copy arduino-builder-arm-1.3.23.tar.bz2 to arduino-builder-linuxarm-1.3.23.tar.bz2 on the download server (and change the build script on jenkins to output the archive with linuxarm for the new releases).

<param name="archive_url" value="https://downloads.arduino.cc/tools/arduino-builder-${platform}-${ARDUINO-BUILDER-VERSION}.tar.bz2" />
<param name="final_folder" value="${staging_folder}/arduino-builder-${platform}/arduino-builder" />
<param name="dest_folder" value="${staging_folder}/arduino-builder-${platform}" />
</antcall>
<copy file="${staging_folder}/arduino-builder-${platform}/arduino-builder" tofile="linux/work/arduino-builder" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong because the file is always copied into tofile="linux/work/arduino-builder" while it should be something like ${staging_folder}/work/arduino-builder. BTW this is still not correct for macosx, probably the best way is to use ${staging_folder}/work/${staging_hardware_folder}/../arduino-builder for everything.

<chmod perm="755" file="linux/work/arduino-builder" />
<move file="${staging_folder}/arduino-builder-${platform}/tools" tofile="linux/work/tools-builder"/>
<copy todir="linux/work/hardware" overwrite="true">
<fileset dir="${staging_folder}/arduino-builder-${platform}/hardware" includes="*.txt"/>
</copy>
<delete dir="${staging_folder}/arduino-builder-${platform}" includeemptydirs="true"/>
</target>

<!-- Set '${dist_file}_available' property if toolchain dist_file is downloaded -->
<!-- Set '${dist_file}_installed' property if toolchain is installed in working directory -->
<!-- hardware/tools/${dist_check_file} is checked for existence -->
Expand Down