Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test device for CI in release process #417

Merged
merged 11 commits into from
Dec 18, 2019
Merged

Add test device for CI in release process #417

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
511eaca
update cloud_test.sh to support more devices
Jun 18, 2019
07540ef
fix test cases
Jun 18, 2019
23ea170
Ignore the unstable test case
Jun 19, 2019
767bb24
Run instrumentation on more devices before release
Jun 19, 2019
dd6235f
Add gCloud login for release
Jun 19, 2019
5029a56
Build test apk for release
Jun 19, 2019
92a6c11
fix unstable test case
Jun 19, 2019
46a9f3e
Merge branch 'master' into kl-add-test-device
Aug 28, 2019
07cc7b8
Merge branch 'master' into kl-add-test-device
nkukday Dec 16, 2019
030761d
Merge branch 'master' into kl-add-test-device
nkukday Dec 17, 2019
f7c4215
fix AlarmMangerInstrumentationTest for API < 19
nkukday Dec 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
32 changes: 32 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,38 @@ jobs:
IS_LOCAL_DEVELOPMENT: false
steps:
- checkout
- run:
name: Build testapp APK
command: |
./gradlew accessToken
./gradlew app:assembleDebug
- run:
name: Build release to test ProGuard rules
command: ./gradlew app:assembleRelease
- run:
name: Build all the test APK
command: ./gradlew assembleAndroidTest
harvsu marked this conversation as resolved.
Show resolved Hide resolved
- run:
name: Log in to Google Cloud Platform
shell: /bin/bash -euo pipefail
command: |
echo "${GCLOUD_SERVICE_ACCOUNT_JSON}" > secret.json
gcloud auth activate-service-account --key-file secret.json --project mapbox-events-android
rm secret.json
- run:
name: Run libcore instrumentation tests on Firebase with more devices
no_output_timeout: 20m
command: |
build_dir="libcore/build"
test_apk_path="outputs/apk/androidTest/debug/libcore-debug-androidTest.apk"
./cloud_test.sh $build_dir $test_apk_path $CIRCLE_BUILD_NUM true
- run:
name: Run libtelemetry instrumentation tests on Firebase with more devices
no_output_timeout: 20m
command: |
build_dir="libtelemetry/build"
test_apk_path="outputs/apk/androidTest/full/debug/libtelemetry-full-debug-androidTest.apk"
./cloud_test.sh $build_dir $test_apk_path $CIRCLE_BUILD_NUM true
- run:
name: Generate Maven credentials
shell: /bin/bash -euo pipefail
Expand Down
37 changes: 33 additions & 4 deletions cloud_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,44 @@ build_dir="$1"
module=$(echo "$build_dir" | cut -d "/" -f1)
test_apk_path="$2"
results_dir="$3"
is_release="$4"

devices="--device model=hammerhead,version=21,locale=en,orientation=portrait \
--device model=hammerhead,version=23,locale=en,orientation=landscape \
--device model=sailfish,version=26,locale=en,orientation=portrait \
--device model=sailfish,version=28,locale=en,orientation=portrait \
"
if [ "$is_release" == "true" ]; then
echo "Is release, adding more devices"
devices="$devices
harvsu marked this conversation as resolved.
Show resolved Hide resolved
--device model=g3,version=19,locale=en,orientation=portrait \
--device model=Nexus6,version=21,locale=en,orientation=portrait \
--device model=Nexus6,version=22,locale=en,orientation=portrait \
--device model=Nexus6,version=23,locale=en,orientation=portrait \
--device model=j1acevelte,version=22,locale=en,orientation=portrait \
--device model=sailfish,version=25,locale=en,orientation=portrait \
--device model=sailfish,version=27,locale=en,orientation=portrait \
--device model=starqlteue,version=26,locale=en,orientation=portrait \
--device model=taimen,version=26,locale=en,orientation=portrait \
--device model=taimen,version=27,locale=en,orientation=portrait \
--device model=walleye,version=26,locale=en,orientation=portrait \
--device model=walleye,version=27,locale=en,orientation=portrait \
--device model=walleye,version=28,locale=en,orientation=portrait \
--device model=zeroflte,version=23,locale=en,orientation=portrait \
--device model=m0,version=18,locale=en,orientation=portrait \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add this to the default list to address #412

--device model=hero2lte,version=23,locale=en,orientation=portrait \
--device model=cheryl,version=25,locale=en,orientation=portrait \
--device model=HWMHA,version=24,locale=en,orientation=portrait \
--device model=FRT,version=27,locale=en,orientation=portrait \
"
fi

gcloud firebase test android models list
gcloud firebase test android run --type instrumentation \
--app app/build/outputs/apk/full/debug/app-full-debug.apk \
--test "${build_dir}/${test_apk_path}" \
--results-dir="$results_dir" \
--device model=hammerhead,version=21,locale=en,orientation=portrait \
--device model=hammerhead,version=23,locale=en,orientation=landscape \
--device model=sailfish,version=26,locale=en,orientation=portrait \
--device model=sailfish,version=28,locale=en,orientation=portrait \
$(echo $devices) \
--environment-variables coverage=true,coverageFile="/sdcard/${module}_coverage.ec" \
--directories-to-pull /sdcard \
--timeout 20m
Expand Down
8 changes: 6 additions & 2 deletions ...idTest/java/com/mapbox/android/core/crashreporter/CrashReportBuilderInstrumentedTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

import android.content.Context;
import android.support.test.InstrumentationRegistry;

import org.json.JSONException;
import org.json.JSONObject;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
Expand Down Expand Up @@ -41,9 +44,10 @@ public void invalidBodyfromJson() {
}

@Test
public void validBodyfromJson() {
public void validBodyfromJson() throws JSONException {
CrashReport report = CrashReportBuilder.fromJson(validJson);
assertEquals(validJson.trim(), report.toJson());
JSONObject jsonValidJson = new JSONObject(validJson);
assertEquals(jsonValidJson.toString(), report.toJson());
}

@Test
Expand Down
11 changes: 7 additions & 4 deletions ...src/androidTestFull/java/com/mapbox/android/telemetry/AlarmMangerInstrumentationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
import android.app.AlarmManager;
import android.content.Context;
import android.content.Intent;
import android.os.Build;
import android.support.test.InstrumentationRegistry;

import org.junit.Assert;
import org.junit.Ignore;
import org.junit.Test;

import java.util.concurrent.CountDownLatch;
Expand All @@ -18,6 +20,7 @@
public class AlarmMangerInstrumentationTest {

@Test
@Ignore
Copy link
Contributor

@nkukday nkukday Nov 25, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not ignore, scheduleExact(...) checks for >= API 19 or returns false since it uses setExact(...), we can use set(...) for < API 19. Let's also restrict this API for tests, something like

  ...
  @RestrictTo(RestrictTo.Scope.TESTS)
  boolean scheduleExact(long interval) {
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
      manager.setExact(AlarmManager.ELAPSED_REALTIME, SystemClock.elapsedRealtime() + interval,
          pendingIntent);
    } else {
      manager.set(AlarmManager.ELAPSED_REALTIME, SystemClock.elapsedRealtime() + interval,
          pendingIntent);
    }
    return true;
  }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this guarantee the test to pass? This is because the test asserts the alarm at definite time and documentation for set() method does not talk about accuracy?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set became inexact in API 19 and setExact was added so prior to 19 set should work.
https://developer.android.com/reference/android/app/AlarmManager#set(int,%20long,%20android.app.PendingIntent)
Docs - Applications whose targetSdkVersion is before API 19 will continue to get the previous alarm behavior: all of their scheduled alarms will be treated as exact.

public void checksAlarmCancelledProperly() throws InterruptedException {
final CountDownLatch latch = new CountDownLatch(2);
final AtomicReference<Integer> broadcastTrack = new AtomicReference<>();
Expand All @@ -39,9 +42,8 @@ public void checksAlarmCancelledProperly() throws InterruptedException {
theAlarmSchedulerFlusher.scheduleExact(elapsedMockedTime2);

Assert.assertFalse(latch.await(30, TimeUnit.SECONDS));
int result = broadcastTrack.get();

Assert.assertEquals(1, result);
Assert.assertEquals(new Integer(1), broadcastTrack.get());
}

@Test
Expand All @@ -63,8 +65,9 @@ public void checksScheduleExact() throws Exception {
AlarmReceiver mockedAlarmReceiver = mock(AlarmReceiver.class);
AlarmSchedulerFlusher theAlarmSchedulerFlusher = new AlarmSchedulerFlusher(mockedContext, mockedAlarmManager,
mockedAlarmReceiver);

Assert.assertTrue(theAlarmSchedulerFlusher.scheduleExact(25));
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not required if we fix scheduleExact(...)

Assert.assertTrue(theAlarmSchedulerFlusher.scheduleExact(25));
}
}

private static AlarmReceiver obtainAlarmReceiver(final AtomicReference<Integer> broadcastTrack,
Expand Down