Skip to content
This repository has been archived by the owner on Oct 2, 2023. It is now read-only.

What are the required config for Jira collector #220

Closed
hongluovz opened this issue Dec 14, 2015 · 33 comments
Closed

What are the required config for Jira collector #220

hongluovz opened this issue Dec 14, 2015 · 33 comments

Comments

@hongluovz
Copy link

hongluovz commented Dec 14, 2015

I provided two main config lines in application.properties:

feature.jiraBaseUrl=http://jira.mycompany.com
feature.jiraCredentials=my_jira_username:my_password

I got this error when starting the jira collector. What would be the root cause of this error. Thanks.

2015-12-14 09:45:00,000 INFO  c.c.d.collector.CollectorTask - Running Collector: Jira
2015-12-14 09:45:00,012 INFO  c.c.d.collector.FeatureCollectorTask - Starting Feature collection...
2015-12-14 09:45:00,125 ERROR c.c.d.c.team.TeamDataClientSetupImpl - There was a problem retrieving or parsing data from the local repository while retrieving a max change date
Returning null
2015-12-14 09:45:00,137 ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.NullPointerException: null
        at com.capitalone.dashboard.util.FeatureWidgetQueries.getQuery(FeatureWidgetQueries.java:66) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientImpl.updateTeamInformation(TeamDataClientImpl.java:131) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:99) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:25) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.CollectorTask.run(CollectorTask.java:62) ~[core-1.5.0-SNAPSHOT.jar!/:1.5.0-SNAPSHOT]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_80]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_80]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_80]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [na:1.7.0_80]
@softwareklinic
Copy link

#218 - check this issue - we pretty much configured all the options provided in the sample application.properties for jira collector... but after that ... jira feature collector was giving error for some query - for which we changed the code in jira feature collector... and it was collecting the data... but then again the same or similar issue on the Jira Widget in Hygieia UI... so we are waiting for the error to be clarified./explained/addressed in issue # 218 posted by my colleague...

#218

@amp5208
Copy link
Contributor

amp5208 commented Dec 14, 2015

@softwareklinic , Thank you for reaching out regarding your use of the Jira Feature Collector. Assuming that your access is sufficient to pull all Jira data, the following error message:

There was a problem retrieving or parsing data from the local repository while retrieving a max change date

Indicates that your MongoDB collector database for Jira is missing content that would be pulled via the following method chain: projectRepo.getProjectMaxChangeDate(featureCollectorRepository.findByName("Jira").getId(),featureSettings.getDeltaStartDate());

To me, this seems that either there are no available change date values generated in your Jira Collector MongoDB collection (less likely), or you are missing a start-date-related property setting for the feature context in your application.properties file (more likely). Can you confirm if you have a feature.deltaCollectorItemStartDate=2008-01-01T00:00:00.000000 property in your application.properties (please replace the date with a valid date for your scenario)?

@hongluovz
Copy link
Author

hongluovz commented Dec 14, 2015

I have this default value:

feature.deltaCollectorItemStartDate=2008-01-01T00:00:00.000000

so I need change it. Thanks

@hongluovz
Copy link
Author

hongluovz commented Dec 14, 2015

I have these config now, but still get the same error. Thanks.

feature.deltaCollectorItemStartDate=2013-08-01T00:00:00.000000
feature.deltaStartDate=2015-08-01T00:00:00.000000
feature.masterStartDate=2015-12-01T00:00:00.000000

@amp5208
Copy link
Contributor

amp5208 commented Dec 15, 2015

You may not have Jira data as far back as 2013, and may account for an error. Please attempt more recent start date configurations and let me know if you see any change. If not, I will have to look into this further.

@hongluovz
Copy link
Author

hongluovz commented Dec 16, 2015

I was able to query jira by using jira-cli on the same server, so my jira access credential should be good. Here are all the config I have in application.properties file:

spring.data.mongodb.database=dashboard
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.username=db
spring.data.mongodb.password=dbpass
feature.cron=0 \* \* \* \* *
feature.pageSize=1000
feature.queryFolder=jiraapi-queries
feature.projectQuery=ProjName
feature.deltaCollectorItemStartDate=2015-01-01T00:00:00.000000
feature.jiraBaseUrl=http://ourjira.ourcompany.com
feature.jiraQueryEndpoint=rest/api/2/
feature.jiraCredentials=username:password
feature.deltaStartDate=2015-03-01T00:00:00.000000
feature.masterStartDate=2015-03-01T00:00:00.000000"

Still got the same error:

2015-12-16 12:12:00,000 INFO  c.c.d.collector.CollectorTask - Running Collector: Jira
2015-12-16 12:12:00,008 INFO  c.c.d.collector.FeatureCollectorTask - Starting Feature collection...
2015-12-16 12:12:00,021 ERROR c.c.d.c.team.TeamDataClientSetupImpl - There was a problem retrieving or parsing data from the local repository while retrieving a max change date
Returning null
2015-12-16 12:12:00,023 ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.NullPointerException: null
        at com.capitalone.dashboard.util.FeatureWidgetQueries.getQuery(FeatureWidgetQueries.java:66) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientImpl.updateTeamInformation(TeamDataClientImpl.java:131) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:99) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:25) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.CollectorTask.run(CollectorTask.java:62) ~[core-1.5.0-SNAPSHOT.jar!/:1.5.0-SNAPSHOT]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_80]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_80]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_80]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [na:1.7.0_80]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_80]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_80]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]

@amp5208
Copy link
Contributor

amp5208 commented Dec 17, 2015

Thank you for following up on this. The issue you are seeing may be related to a defect found recently affecting the feature collectors, in general. I am looking into this right now and will push a fix as soon as possible.
@amitmawkin & @tabladrum : Please mark this as a defect, related to the VersionOne Feature Collector defect with $query syntax

@hongluovz
Copy link
Author

thank you.

@softwareklinic
Copy link

Same issue exists in Jira collector as well.

@amp5208
Copy link
Contributor

amp5208 commented Dec 18, 2015

Definitely see this, yes. It seems to arose out of a change from the Java 1.7 to 1.8 framework for the Hygieia project; it would seem that certain repository query syntax I used is no longer supported in that version.

I am actively fixing now; hopefully I will have a fix within the next two weeks. I apologize for any inconvenience this may have caused.

@hongluovz
Copy link
Author

hongluovz commented Dec 18, 2015

Just rebuild on a new server with JDK 1.8, I still got the same error. Thanks.

$ java -version
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
$ uname -a
Linux  3.10.0-229.7.2.el7.x86_64 #1 SMP Fri May 15 21:38:46 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

@amp5208
Copy link
Contributor

amp5208 commented Dec 18, 2015

@hongluovz , Allow me to clarify: The build was moved to a 1.8 JDK platform, which in turn broke the syntax in the FeatureCollector @Query annotations; however, the move to 1.8 may have also been required for some Core changes, and thus the build will no longer propagate as-is for 1.7, which the FeatureWidget code was original built/tested. I am refactoring/making required code changes to ensure that the build works going forward for 1.8.

Until these changes are made, the FeatureCollectors will not build with either 1.7 or 1.8 and are broken. I am working quickly to resolve these issues in the code.

@amp5208
Copy link
Contributor

amp5208 commented Jan 4, 2016

@hongluovz , As a follow-up, I wanted to let you know that a fix for this issue should be coming shortly. I have pushed a fix as a pull request to fix the Feature collector compatibility issues. Once it is fully validated, it should be pulled into Master. You can view the pull request at the following link: #253

@Wills2607
Copy link

@amp5208 , Is this issue got fixed.?

@Wills2607
Copy link

Wills2607 commented Jan 5, 2016

I am getting this error, can you please help.?

-01-05 11:16:00,089 INFO  c.c.d.collector.FeatureCollectorTask - Starting Feature collection...
2016-01-05 11:16:00,255 ERROR c.c.d.c.team.TeamDataClientSetupImpl - There was a problem retrieving or parsing data from the local repository while retrieving a max change date
Returning null
2016-01-05 11:16:00,261 ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.NullPointerException: null
        at com.capitalone.dashboard.util.DateUtil.fromISODateTimeFormat(DateUtil.java:97) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientSetupImpl.getChangeDateMinutePrior(TeamDataClientSetupImpl.java:133) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientImpl.updateTeamInformation(TeamDataClientImpl.java:112) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:97) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:24) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.CollectorTask.run(CollectorTask.java:63) ~[core-1.5.0-SNAPSHOT.jar!/:1.5.0-SNAPSHOT]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_45]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]

@amp5208
Copy link
Contributor

amp5208 commented Jan 5, 2016

@Wills2607 , Thank you for reaching out. A fix has been provided in the above-noted pull request, but is not fully validated/merged yet. Please check back when it is validated and merged. The issue you are seeing is related to the defect or that you are missing a delta start date entry in your property file for the feature collectors.

@Wills2607
Copy link

Wills2607 commented Jan 5, 2016

Still same issue. Delta Date values in my properties file.

feature.deltaStartDate=2016-01-01T00:00:00.000000
feature.masterStartDate=2016-01-01T00:00:00.000000

@amp5208
Copy link
Contributor

amp5208 commented Jan 5, 2016

Are you seeing the issue still with the most recent pull requested version referenced above? As mentioned, there is a defect fix that is currently being implemented which may fix your issue.

@Wills2607
Copy link

Yes - I have download from master repo yesterday, but still I am getting the same error.

@amp5208
Copy link
Contributor

amp5208 commented Jan 6, 2016

This defect fix is not yet in master - this may be your issue. It is still under development, but if you want to validate that my defect fixes also fix your issue, please pull and check this forked version for now: https://github.com/amp5208/Hygieia
Please note: After the fix is merged to Capital One's org master, I will be destroying this fork.

@Wills2607
Copy link

Wills2607 commented Jan 6, 2016

Thank you, that got fixed. now I am getting different error.

2016-01-06 14:07:00,255 ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.NullPointerException: null
        at com.capitalone.dashboard.util.DateUtil.fromISODateTimeFormat(DateUtil.java:97) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientSetupImpl.getChangeDateMinutePrior(TeamDataClientSetupImpl.java:132) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.client.team.TeamDataClientImpl.updateTeamInformation(TeamDataClientImpl.java:113) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:97) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashbo
ard.collector.FeatureCollectorTask.collect(FeatureCollectorTask.java:24) ~[jira-feature-collector.jar!/:1.5.0-SNAPSHOT]
        at com.capitalone.dashboard.collector.CollectorTask.run(CollectorTask.java:63) ~[core-1.5.0-SNAPSHOT.jar!/:1.5.0-SNAPSHOT]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.1.7.RELEASE.jar!/:4.1.7.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java

@Wills2607
Copy link

can I have the working sample properties file ?

@Wills2607
Copy link

Wills2607 commented Jan 6, 2016

amp5208 : can you pl help me if any thing wrong with jira collector properties file.

Properties file:-

feature.pageSize=1000

#
feature.queryFolder=jiraapi-queries
#
feature.storyQuery=story
feature.epicQuery=epic
feature.projectQuery=
feature.memberQuery=
feature.sprintQuery=
feature.teamQuery=
feature.trendingQuery=

#Jira
feature.jiraProxyUrl=
feature.jiraProxyPort=
######################
#
#
feature.sprintDays=60
#
feature.sprintEndPrior=7
######################
#Scheduled
feature.scheduledPriorMin=2
#Delta
#feature.deltaCollectorItemStartDate=2015-03-01T00:00:00.000000
feature.deltaCollectorItemStartDate=2008-01-01T00:00:00.000000

######################
#Spring
spring.data.mongodb.database=xxxx
spring.data.mongodb.host=xxxxxx
spring.data.mongodb.port=xxxxxxx
spring.data.mongodb.username=xxxxxxxx
spring.data.mongodb.password=xxxxxx

#Jira
feature.jiraBaseUrl=http://xxxx.com
feature.jiraQueryEndpoint=rest/api/2/
feature.jiraCredentials=xxxxxx:xxxxxxx
#feature.jiraOauthAuthtoken=oauth,
#feature.jiraOauthRefreshtoken=oauth,
#feature.jiraOauthRedirecturi=uri.this.is.test:uri
#feature.jiraOauthExpiretime=234567890987

######################
#Chron
feature.cron=0

#StartDate
feature.deltaStartDate=2015-03-01T00:00:00.000000

@Wills2607
Copy link

@amp5208 - Any update on this issue?

@amp5208
Copy link
Contributor

amp5208 commented Jan 7, 2016

@Wills2607, Your issue is likely now that you have not updated your property files to handle for the new Jira configurations available in that version. Please see the following link and ensure that you have the following feature.jira* properties: https://github.com/amp5208/Hygieia/blob/master/jira-feature-collector/src/main/resources/jira-feature-collector.properties

feature.jiraIssueTypeId=
feature.jiraSprintDataFieldName=
feature.jiraEpicIdFieldName=

Additionally, I noticed in your above-noted property file that your Jira credentials property was malformed. Jira credentials should be in username:password format, converted to a base64 string. Please verify and try again.

Note: You can convert to a base64 string easily with a Mac using the following command:

echo "username:password" | base64

@hongluovz
Copy link
Author

hongluovz commented Jan 9, 2016

@amp5208 Should we use applicattion.properties file to configure these attributes I instead of using jira-feature-collector.properties?

I finally got some jira projects back from our jira server, but always got only 5 projects although we have a lot more. These 5 projects were also duplicated many times in mongodb.
Below are some of my config in application.property file. Can I change the team or projectQuery to filter out some projects?

#Convert above username:password into e64 bit string
feature.jiraCredentials=U1ZDLWh5Z2llaWEtamlyYTplVDIK
#In-built folder housing prepared REST queries (required)
feature.queryFolder=jiraapi-queries

#Jira API Query file names (String template requires the files to have .st extension) (required)
feature.storyQuery=story
feature.epicQuery=epic
feature.projectQuery=projectinfo
feature.memberQuery=memberinfo
feature.sprintQuery=sprintinfo
feature.teamQuery=teaminfo
feature.trendingQuery=trendinginfo"

Thanks.

@amp5208
Copy link
Contributor

amp5208 commented Jan 11, 2016

So long as you run the Jar with the name and location Springboot -D flags to reference your properties file, it does not matter if you place these arguments in the global application.properties or in a local jira-feature-collector.properties file.

However, please note, we are still fixing some bugs with this version that you are attempting to work with; until it is merged with master, it is considered volatile. In particular, this working version is seeing the issue you described, where duplicate ScopeOwners (e.g., projects) will appear.

Lastly, you may want to remove the 64bit encoded string in your last comment...

@amp5208
Copy link
Contributor

amp5208 commented Jan 14, 2016

Fix is ready for your originally-noted issue - pending merge of #253 pull request before closing this issue. Once merged, please pull from master and use any new property file settings noted in the Feature content read-me's.

@amp5208
Copy link
Contributor

amp5208 commented Jan 14, 2016

This issue has been fixed. @tabladrum or @amitmawkin , Please feel free to close this issue.

@SubramaniMurthy
Copy link

Hi,
I have configured JIRA Collector. And, collector is not throwing any error when i executing it.
It displays Agile Content Tool Type: Jira and all Project Names from JIRA tool in Feature Widget.

But, after selecting a project and saving the widget, it displays as SPRINT: [NO SPRINT AVAILABLE] and everything with '0' value.

Would like to understand what would i have missed in the properties file. Please HELP !!!

Thanks,
Subbu

@amp5208
Copy link
Contributor

amp5208 commented Feb 26, 2016

@SubramaniMurthy , Thank you for reaching out, and I apologize for the delay. I noticed this, myself, after a revert and re-merge activity that happened recently. I believe that something in the API and/or Core was not properly pushed upstream during a pull request merge.

I plan on following up with this item as soon as possible and as soon as priorities allow me.

To verify, however, can you confirm with me your data within your MongoDB store? Are you seeing data that should other wise map to the following scenario: An active Issue mapped to a status, mapped to an active Project, that is mapped to an active, in-progress Sprint? To validate this in the data, please check you feature MongoDB collection based on your Jira collector ID.

@amp5208
Copy link
Contributor

amp5208 commented Mar 11, 2016

@SubramaniMurthy, This issue has been continued in the following Issue post: #276. I have also provided a concise "working" property example in that post. Please reference that post for more information.

@tabladrum , This post can be closed in lieu of the related issue.

@ghost
Copy link

ghost commented Dec 14, 2016

credential is not working in JIRA , how can resolve

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants