Skip to content

Use case test results and problems

Jump to bottom
H. Joe Lee edited this page Mar 19, 2017 · 45 revisions

Hyrax fails to produce correct outputs occasionally. This table will record the instances.

Date Test Failure Note
11/24/2016 use case 7 10+
11/29/2016 all 0
12/06/2016 use case 6 1 11:02 CST
12/06/2016 all 0 12:00 CST
12/11/2016 all 0 04:00 CST
12/18/2016 use case 7 1 02:22 PST
12/25/2016 all 0 04:00 CST
01/08/2017 all 0 04:00 CST
01/23/2017 all 0 12:00 CST
02/05/2017 use case 3 3 04:00 CST - data reading for default handler failed
02/19/2017 all 0 04:00 CST - all 15 tests ended 22:07 CST.
02/26/2017 all 0 04:00 CST - all 17 tests ended on 2/27/2017 01:58 CST.
02/28/2017 all 0 00:08 CST - all 17 tests ended on 2/28/2017 23:51 CST with overlap between Arch1 and Arch2.
03/05/2017 all 0 04:00 CST - all 17 tests ended on 3/06/2017 02:13 CST.

Performance of Arch #1 vs. #2 vs. #3

Date Test Arch #1 Arch #2 Arch #3 Note
03/13/2017 use case 21 1h15m41s 3h37m28s 5h17m12s 1EC2
03/13/2017 use case 20 1h8m12s 4h55m55s 6h47m24s 1EC2
03/13/2017 use case 19 1h4m55s 1h50m1s 3h0m29s 1EC2
03/12/2017 use case 18 44m53s 1h30m59s 2h21m53s 1EC2
03/12/2017 use case 17 aggA 1h0m35s 6m5s 5m49s 1EC2
03/12/2017 use case 16 aggM 26m46s 6m18s 6m46s 1EC2
03/12/2017 use case 15 aggA 9h42m24s 31m36s 34m36s 1EC2
03/12/2017 use case 14 aggM 3h8m10s 33m8s 34m10s 1EC2
03/12/2017 use case 13 aggM 18m34s 8m44s 11m6s 1EC2
03/12/2017 use case 12 aggM 18m51s 4m58s 4m12s 1EC2
03/12/2017 use case 11 aggA 59m10s 6m13s 7m13s 1EC2
03/12/2017 use case 10 aggA 57m19s 3m6s 2m55s 1EC2
03/12/2017 use case 7 aggA 59m16s 2m21s 2m13s 1EC2
03/12/2017 use case 6 aggM 19m33s 3m40s 3m5s 1EC2
03/12/2017 use case 2 D4SS 0.211s 0.782s 0.797s 1EC2
03/12/2017 use case 2 DAP4 0.246s 1.208s 1.086s 1EC2
03/12/2017 use case 2 DMR 14.547s 0.779s 0.783s 1EC2
03/05/2017 use case 17 aggA 58m28s 6m11s 6m19s 1EC2
03/05/2017 use case 16 aggM 19m4s 6m17s 5m46s 1EC2
03/05/2017 use case 15 aggA 10h20m8s 29m17s 38m27s 1EC2
03/05/2017 use case 14 aggM 3h12m48s 32m31s 30m41s 1EC2
03/05/2017 use case 13 aggM 18m18s 9m21s 10m57s 1EC2
03/05/2017 use case 12 aggM 18m19s 4m59s 4m27s 1EC2
03/05/2017 use case 11 aggA 56m57s 6m26s 7m18s 1EC2
03/05/2017 use case 10 aggA 56m23s 3m13s 2m53s 1EC2
03/05/2017 use case 7 aggA 57m13s 2m26s 2m10s 1EC2
03/05/2017 use case 6 aggM 19m03s 3m59s 3m8s 1EC2
03/05/2017 use case 2 D4SS 0.210s 0.957s 0.791s 1EC2
03/05/2017 use case 2 DAP4 0.221s 1.054s 0.957s 1EC2
03/05/2017 use case 2 DMR 9.126s 0.828s 0.768s 1EC2
03/04/2017 All for Arch #3 n/a n/a 1h48m Started at 4:08am CST
03/03/2017 All for Arch #2 n/a 1h40m n/a Started at 4:08am CST
03/02/2017 All for Arch #1 19h n/a n/a Started at 4:08am CST
02/28/2017 All for Arch #3 n/a n/a 1h43m Started at 22:08am CST
02/28/2017 All for Arch #2 n/a 1h51m n/a Started at 20:08am CST
02/28/2017 All for Arch #1 20h44m n/a n/a Started at 00:08am CST
02/26/2017 use case 17 aggA 58m31s 5m52s 5m51s 1EC2
02/26/2017 use case 16 aggM 18m51s 16m15s 15m41s 1EC2
02/26/2017 use case 15 aggA 10h6m7s 29m28s 27m26s 1EC2
02/26/2017 use case 14 aggM 3h14m44s 32m7s 30m7s 1EC2
02/26/2017 use case 13 aggM 19m19s 9m19s 11m01s 1EC2
02/26/2017 use case 12 aggM 18m55s 5m 4m30s 1EC2
02/26/2017 use case 11 aggA 59m7s 6m9s 7m21s 1EC2
02/26/2017 use case 10 aggA 57m5s 3m8s 2m51s 1EC2
02/26/2017 use case 7 aggA 58m48s 2m23s 2m14s 1EC2
02/26/2017 use case 6 aggM 18m51s 3m46s 3m6s 1EC2
02/26/2017 use case 2 D4SS 0.231s 0.784s 0.777s 1EC2
02/26/2017 use case 2 DAP4 0.232s 0.995s 1.024s 1EC2
02/26/2017 use case 2 DMR 9.558s 0.835s 0.778s 1EC2
02/25/2017 All for Arch #3 n/a n/a 1h46m Started at 4:08am CST
02/24/2017 All for Arch #2 n/a 1h40m n/a Started at 4:08am CST
02/23/2017 All for Arch #1 18h42m n/a n/a Started at 4:08am CST
02/19/2017 use case 15 aggA 9h45m21s 31m19s 35m56s 1EC2
02/19/2017 use case 14 aggM 3h13m11s 31m43s 32m18s 1EC2
02/19/2017 use case 13 aggM 18m16s 9m32s 11m15s 1EC2
02/19/2017 use case 12 aggM 18m35s 4m56s 4m31s 1EC2
02/19/2017 use case 11 aggA 1h1m1s 6m30s 7m39s 1EC2
02/19/2017 use case 10 aggA 1h10s 3m9s 3m5s 1EC2
02/19/2017 use case 7 aggA 59m31s 2m26s 2m12s 1EC2
02/19/2017 use case 6 aggM 19m16s 3m43s 3m6s 1EC2
02/16/2017 use case 15 aggA n/a 32m36s 38m08s 1EC2
02/15/2017 use case 14 aggM n/a 8h11m11s 1h27m51s 1EC2
02/15/2017 use case 13 aggM n/a 9m17s 11m29s 1EC2
02/15/2017 use case 12 aggM n/a 5m2s 4m39s 1EC2
02/15/2017 use case 11 aggA n/a 6m6s 7m34s 1EC2
02/15/2017 use case 10 aggA n/a 3m17s 3m07s 1EC2
02/07/2017 use case 6 aggM 18m05s 3m48s 3m13s 1EC2
02/07/2017 use case 7 aggA 56m34s 2m34s 2m17s 1EC2
02/05/2017 use case 6 aggM 17m53s 3m48s 3m5s 3EC2
02/05/2017 use case 7 aggA 54m29s 2m22s 2m4s 3EC2
02/03/2017 use case 2 DMR 5.527s 1.18s 1.39s 3EC2
02/03/2017 use case 2 DAP4 0.443s 1.064s 1.127s 3EC2
02/03/2017 use case 2 D4SS 0.273s 0.803s 0.821s 3EC2
02/03/2017 use case 6 aggM 3m11s
02/03/2017 use case 7 aggA 2m12s
  • 3EC2: 3 EC2 instances were used. Arch #3 EC2 instance runs the latest Hyrax code.
  • 1EC2: 1 EC2 instance was used. All 3 architectures run on the same host.

Performance of Arch #1 vs. #2

Date Test Arch #2 Arch #1 Note
02/05/2017 use case 2 DMR 1.282s 0.548s 4:00am
02/05/2017 use case 2 DAP4 1.094s 0.394s 4:00am
02/05/2017 use case 2 D4SS 0.798s 0.312s 4:00am
01/25/2017 use case 6 aggM 3m 23s 18m dmrpp+bln
01/25/2017 use case 2 DMR 0.808s 0.742s dmrpp
01/25/2017 use case 2 DAP4 1.093s 0.720s dmrpp
01/25/2017 use case 2 D4SS 0.790s 0.545s dmrpp
01/25/2017 use case 7 aggA 2m 24s 57m dmrpp+bln
01/25/2017 use case 6 aggM 4m 18m dmrpp+bst
01/23/2017 use case 1 DAS 0.621s 2.440s weekly #1
01/23/2017 use case 1 DDS 0.160s 2.184s weekly #1
01/23/2017 use case 1 DODS 0.246s 0.313s weekly #1
01/23/2017 use case 1 SS 0.103s 0.136s weekly #1
01/23/2017 use case 2 DMR 0.318s 14.804s weekly #1
01/23/2017 use case 2 DAP4 0.289s 0.401s weekly #1
01/23/2017 use case 2 D4SS 0.222s 0.306s weekly #1
01/23/2017 use case 3 DAS 0.260s 0.257s #2 use c1
01/23/2017 use case 3 DDS 0.159s 0.153s #2 use c1
01/23/2017 use case 3 DODS 0.231s 0.233s #2 use c1
01/23/2017 use case 3 SS 0.101s 0.103s #2 use c1
01/23/2017 use case 1 DAS 0.239s 0.347s
01/23/2017 use case 1 DDS 0.153s 0.223s
01/23/2017 use case 1 DODS 0.237s 0.346s
01/23/2017 use case 1 SS 0.100s 0.136s
01/23/2017 use case 2 DMR 0.062s 0.479 s dmrpp
01/23/2017 use case 2 DAP4 0.056s 0.447s dmrpp
01/23/2017 use case 2 D4SS 0.056s 0.375s dmrpp
  • weekly #1: Weekly test was running on Arch #1 server during the test.
  • #2 use c1: Arch #2 used the use case #1 (CF=true option).
  • dmrpp: Arch #2 used dmrpp resource instead of S3 gateway.
  • drmpp+bst: Arch #2 used dmrpp resource and besstandalone. Arch #1 used bescmdln.
  • drmpp+bln: Arch #2 used dmrpp resource and bescmdln. Arch #1 also used bescmdln.

Clone this wiki locally