-
Notifications
You must be signed in to change notification settings - Fork 0
/
estimates.txt
540 lines (401 loc) · 15.9 KB
/
estimates.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
8th January:-
Time sheet:- 30 mins
SQNKS-198:- notor_customer_engage_pct showing error on visits (aka no data) for dealer 5704.
understanding ticket.
its because it is opt_out and its notor status is 3.So, its data is not collected and opt_out date is 20180322.
Investigated nciautoupdate_main failure(service retention daily.)
Added manual step for SPAAutoUpdate_Notor job:- 20 mins.
Added and ran manual step for DE-2693 and DE-2694.
DE-2503:- Break split.json per release/hotfix
Made separate steps file for release-20190111.
Tested changes
Resolved conflicts.
ACCS18-35:- Mirrorfiles process for VPC data extract
check if current keys are working for decryption of the pgp file.
and what we need in order to decrypt it.
Tring different to open the file.
Sqnks standup:- 1 hour
Resolve conflicts for DE-2503.
9th January:-
ACCS18-35:- added script call.
ACCS18-36:- Importing VPC data extracts
Understanding ticket.
Understanding file structure.
Understanding transform script by joachim
understanding script bpcWrapper.sh
understanding script convert_fixed_to_delimited.sh
understanding script bulkloadwrapper.sh
coding
testing
Looker Insight Automation testing.
SQNKS stand up:- 1 hour
10th January:-
checking mails.
Add column names for ACCS18-36 import.(asked by joachim)
Make changes for ACCS18-35.(asked by joachim)
sqnks-55:-
uninstall datagrip
install datagrip
setup athena
check if the table exists.
MAFY18-140:- AGP/IPRO Corporate Quarterly Report.
Understanding scripts(apg_irpo_corporate_report)
Make sql changes.
Make pyhton changes
Testing.
sqnks stand up:- 1 hour 10 mins
11th January:-
Checked AppAutoUpdate.
Looker Insights testing
14th January:-
Looker Insights testing
15th January:-
Looker Insights testing(Looker_Connection.py)
16th January:-
Final testing.
AS18-466:- Understanding Ticket.(Load process for division, region, area, and district financial data (aka composite data))
coding
testing
MAFY18-140:- review points()AGP/IPRO Corporate Quarterly Report
testing.
SQNKS-240:-(Find Polk Loyalty Data feed from polk sftp server and mirror it)
find password.
find feed.
mirror it.
discussion with carlos and michael
17th January:-
SQNKS-240:-Find Polk Loyalty Data feed from polk sftp server and mirror it.
sqnks-238:- test soa metrics.
sqnks-127:-NMAC: Create App Packet
understanding scripts.
insight looker Automation testing:- discussion with andrea and carlos.
18th January:-
sqnks-127:-NMAC: Create App Packet.
code.
find tables.
write sql for tables.
test.
sqnks-254:-
Finding reason for the cause.
sqnks-256:-
Investigated job.
as18-465:-
code
find password for pgp decrypt
test
21st January:-
build runner dashboard
nagbot messages
AS18-465:-
write mirror and import calls
SQNKS-254:-
commented out retry logic.
tested code.
SQNKS-265:-
find the cause.
find modification time issue(in mirror files)
quarantine feeds.
SQNKS-266:-
find the cause.
SQNKS-267:-
Investigated ES_Monthly_Reports
Investigated SPAAutoUpdate_STEP_SEND_THIRDPARTY(stuck due to AppAutoUpdate)
Investigated CloseNagBotEvents
Investigated ResendNotifications
SQNKS-268:-
Investigated trendi
Investigated NMAC_Recommended Metrics
Investigated Update_PDM_Metrics
resolve release issue.
ACCS18-38:-
find cause
fixed and tested changes.
22nd January:-
AS18-455:-
worked on review points
tested changes
AS18-466:-
worked on review points
tested changes
nagbot messages
build runner dashboard
prd failures:-
stg failures:-
Investigated SPAAutoUpdate_STEP_GET(failing due to decryp issue, has been fixed on dev)
Investigated ES_Monthly_Reports(could not generate report on excel stage server)
Investigated NNA_FIN(has not run because of failure of SPAAutoUpdate_STEP_GET)
dev failure:-
to be investigated:-
SQNKS-230:- Investigated CloseNagBotEvents(failing due to snowflake package issue)
SQNKS-230:- Investigated ResendNotifications(failing due to snowflake package issue)
Investigated Update_PDM_Metrics
SQNKS-271:- Investigated SPAAutoUpdate_STEP_DATAIMPORT
Unexpected file:-
Infiniti_Certification_20190121.csv
Looker Insights Automation Testing
23rd January:-
ACCS18-38:-
Updated metadata for feed NNA2SQRT*
nagbot notifications
build runner dashboard
Unexpected Files:-
NNA_Infiniti_Monthly_Service_201901200900.txt(seems to be a new feed)
NNA_Infiniti_Monthly_Sales_201901200900.txt(seems to be a new feed)
Infiniti_AllFields_woNatl.xlsx(seems to be a new feed)
Infiniti_Certification_20190122.csv(seems to be a new feed)
SQNKS-281:- nna_org_2019-01-22.csv.pgp failed extraction
Remote path:/nna_peoplesoft/inbound/hr/prod/v2
Failures:-
prd:-
SQNKS-280:- nciautoupdate_main()
inform Mark about NNA_FIN(Assertion was the last step of job)
stg:-
ES_Monthly_Reports
nciautoupdate_main
SQNKS-282:- SPAAutoUpdate_STEP_DATAIMPORT
SQNKS-230:- Investigated CloseNagBotEvents and ResendNotifications(failing due to snowflake package issue)
dev:- nciautoupdate_main
SQNKS-127:-
24th January:-
Build runner dashboard
nagbot notifications
Failures:-
prd:-
stg:-
ES_Monthly_Reports
Reason:- Could not generate excel report on excel-stg-1.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-267
CloseNagBotEvents and ResendNotifications:
Reason:- Failed due to a missing package snowflake
Ticket:- https://square-root.atlassian.net/browse/SQNKS-230
dev:-
Update_NCI_Metrics, PDMAutoUpdate_PROPAGATE, Update_Jenkins_Metrics, Update_NMAC_Metrics, Update_Demo_Metrics
Reason:-
There was a permission denied error for a relation test on pg-metrics-dev-1.
Action taken:-
Assuming someone made it for testing purpose and forgot to drop.
I dropped relation test.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-286
Unexpected files:-
NNA_Infiniti_Monthly_Service_201901230900.txt(seems to be a new file)
Failed sniffes:-
NISSAN_DATAFEED_R3_201512.csv
Ticket:- https://square-root.atlassian.net/browse/SQNKS-287
InfPMAPenetration_R12_20190123.txt.PGP
Ticket:- https://square-root.atlassian.net/browse/SQNKS-288
SQNKS-275:-
Investigated the issue.(Phone number not showing for dealers)
SQNKS-127:-NMAC App Packet
25th January:-
nagbot notifications
build runner dashboard
Failures:-
prd:-
NMACAutoUpdate:-
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-291
stg:-
CloseNagBotEvents and ResendNotifications:
Reason:- Failed due to a missing package snowflake
Ticket:- https://square-root.atlassian.net/browse/SQNKS-230
ES_Monthly_Reports
Reason:- Could not generate excel report on excel-stg-1.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-267
NMACAutoUpdate:-
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-291
dev:-
NMACAutoUpdate:-
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-291
AS18-465 and AS18-466:- Testing
Could not load large files.
Working on importing large files.
made code changes
tested changes
ACCS18-41:-
Understanding Ticket.
28th January:-
Nagbot notifications
Build runner dashboard
`Failures:-`
`prd`:-
`ESAnalytics_Report`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-293
`SPAAutoUpdate_STEP_INTERMED`:-
Reason:- Data for dealer 70556 was incomplete in dealer data feed.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-294
`stg`:-
`CloseNagBotEvents and ResendNotifications:`
Reason:- Failed due to a missing package snowflake
Ticket:- https://square-root.atlassian.net/browse/SQNKS-230
`ES_Monthly_Reports`
Reason:- Could not generate excel report on excel-stg-1.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-267
`AGP_IPRO`
Reason:- Data was updated unexpectedly for 2 of the dealers(Assertion Failure)
Ticket:- https://square-root.atlassian.net/browse/SQNKS-295
`SPAAutoUpdate_STEP_INTERMED`:-
Reason:- Data for dealer 70556 was incomplete in dealer data feed.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-294
`dev:-`
`AGP_IPRO`
Reason:- Data was updated unexpectedly for 2 of the dealers(Assertion Failure)
Ticket:- https://square-root.atlassian.net/browse/SQNKS-295
`SPAAutoUpdate_STEP_INTERMED:-`
Reason:- Data for dealer 70556 was incomplete in dealer data feed.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-294
`Update_PDM_Metrics:-`
Reason:-
Ticket:-
ACCS18-41:- Code.
Testing.
SQNKS-127:- Testing.
Unexpected Files:-
Infiniti_Certification_20190124.csv(seems to be a new feed)
NNA_Infiniti_Monthly_Service_201901240900.txt(seems to be a new feed)
NCI_Service_Retention_201901_20190121.csv(add it in metadata)
29th January:-
Nagbot notifications
build runner dashboard
`prd`
`Latefiles_Notifications`
Reason:- Job was not able to connect to pg-dev-vw-1
Ticket:- https://square-root.atlassian.net/browse/SQNKS-298
`ESAnalytics_Report`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-293
`stg`
`Latefiles_Notifications`
Reason:- Job was not able to connect to pg-dev-vw-1
Ticket:- https://square-root.atlassian.net/browse/SQNKS-298
`ES_Monthly_Reports`
Reason:- Could not generate excel report on excel-stg-1.
Ticket:- https://square-root.atlassian.net/browse/SQNKS-267
`dev`
`Latefiles_Notifications`
Reason:- Job was not able to connect to pg-dev-vw-1
Ticket:- https://square-root.atlassian.net/browse/SQNKS-298
`VWAutoUpdate`
Reason:- No space left on device
Ticket:- https://square-root.atlassian.net/browse/SQNKS-299
`Unexpected files`
Infiniti_Certification_20190126.csv(seems to be new feed)
AS18-466:-
Minor fix(rearranging of columns)
P:-
Keep an eye on NCI retention file as asked by Qiong
Looker insight testing tool.
testing of SQNKS-127.
MAFY18-144:-
Understanding Ticket
making changes
30th January:-
nagbot notifications
build runner dashboard
`Build Failures`
`prd`
`ESAnalytics_Report`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-293
`AGP_IPRO`
Reason:- Assertion Failure, PSAT data updated unexpectedly
Ticket:- https://square-root.atlassian.net/browse/SQNKS-306
`stg`
`AGP_IPRO`
Reason:- Assertion Failure, PSAT data updated unexpectedly
Ticket:- https://square-root.atlassian.net/browse/SQNKS-306
`dev`
`AGP_IPRO`
Reason:- Assertion Failure, PSAT data updated unexpectedly
Ticket:- https://square-root.atlassian.net/browse/SQNKS-306
MAFY18-144:-
Completed it.
INF18-15:-
Understand
code changes
metadata changes
test
SQNKS-127:-
Testing
Finding files which are required.
P:-
Handled NNA_FIN
31st January:-
`Build Failures`
`prd`
`ESAnalytics_Report`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-293
`PartsAutoUpdate_PartSales`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`NNA_FIN`
Reason:- Assertion failure, Dealer count less than 95 percent(New feed didn't load properly)
Ticket:- https://square-root.atlassian.net/browse/SQNKS-310
`SPAAutoUpdate_STEP_DATAIMPORT`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`stg`
`PartsAutoUpdate_PartSales`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`NNA_FIN`
Reason:- Assertion failure, Dealer count less than 95 percent(New feed didn't load properly)
Ticket:- https://square-root.atlassian.net/browse/SQNKS-310
`SPAAutoUpdate_STEP_DATAIMPORT`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`SPAAutoUpdate_STEP_LOOKER_DATA_GENERATION`
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-314
`dev`
`PartsAutoUpdate_PartSales`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`NNA_FIN`
Reason:- Assertion failure, Dealer count less than 95 percent(New feed didn't load properly)
Ticket:- https://square-root.atlassian.net/browse/SQNKS-310
`SPAAutoUpdate_STEP_DATAIMPORT`
Reason:- Bad feed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-311
`SPAAutoUpdate_STEP_LOOKER_DATA_GENERATION`
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-314
Tickets to work on:-
SQNKS-310:-
steps to do for it on stg and prd after job runs successful on dev
quarantine files mirrored yesterday
delete data from tables manually
delete entry from FinancialDataFilesLoaded.txt
SQNKS-266:- Done
MAFY18-144:- Done
INF18-15:- Done
MAA-59:- Done
SQNKS-311:- Done
1st Feb:-
Nagbot notifications
Build Runner dashboard
`Build Failures`
`prd`
`ESAnalytics_Report`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-293
`AGP_IPRO`
Reason:-
Ticket:- https://square-root.atlassian.net/browse/SQNKS-355
`stg`
`dev`
`AGP_IPRO`
Reason:- process killed
Ticket:- https://square-root.atlassian.net/browse/SQNKS-355
SQNKS-238:-
Refactoring
P:-
handled ResendNotifications and CloseNagBotEvents on dw-admin-prd-1
handled AGP_IPRO_Reports on stg
to-do:-
Get INF18-15 reviewed from abhishek and add its cherry-pick
to-do:-
Close MainFrameAutUpdate tickets(blocking jobs)