Skip to content

FINERACT-2421: Optimize github actions workflow#5799

Open
adamsaghy wants to merge 1 commit intoapache:developfrom
adamsaghy:FINERACT-2421/optimize-github-actions-workflow
Open

FINERACT-2421: Optimize github actions workflow#5799
adamsaghy wants to merge 1 commit intoapache:developfrom
adamsaghy:FINERACT-2421/optimize-github-actions-workflow

Conversation

@adamsaghy
Copy link
Copy Markdown
Contributor

Description

Describe the changes made and why they were made. (Ignore if these details are present on the associated Apache Fineract JIRA ticket.)

Checklist

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Write the commit message as per our guidelines
  • Acknowledge that we will not review PRs that are not passing the build ("green") - it is your responsibility to get a proposed PR to pass the build, not primarily the project's maintainers.
  • Create/update unit or integration tests for verifying the changes made.
  • Follow our coding conventions.
  • Add required Swagger annotation and update API documentation at fineract-provider/src/main/resources/static/legacy-docs/apiLive.htm with details of any API changes
  • This PR must not be a "code dump". Large changes can be made in a branch, with assistance. Ask for help on the developer mailing list.

Your assigned reviewer(s) will follow our guidelines for code reviews.

@adamsaghy adamsaghy force-pushed the FINERACT-2421/optimize-github-actions-workflow branch from 201d575 to 2d12d94 Compare April 23, 2026 15:21
Comment thread .github/workflows/build-mariadb.yml Outdated

jobs:
test:
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cant seem to agree that we should add this here on existing checks

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed since all dependent workflow must wait on build-core to be finished.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please confirm me What am i seeing that you are trying to cache the Tests on Push, What would happen if Someone does some major work (new feature or changes) in their forked Branch and then raise PR like every PR. Will that work on that. Im seeing it can fail on forked repo with new features. LIke we are not caching e2e tests from forked repo due to security reason.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not caching on push anything.

Build-core compiles and generates required files. Once everything built, we are storing in cache the built files.

All workflow depends on build-core so that run first. Once it's finished, we are fetching the built files (compiles classes, generated fineract-client, etc.) from the cache and run the tests, build docker image, etc.

Why the change?

  • All workflow was independent and standalone which means ALL of them were compiling and regenerating files which is unnecessary and led to java heap space issues.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, So is it like this?

1st -> Build Core will run it will Cache the Compiled files.
2nd -> Other tests will run taking on build files (Cucumber e2e test. and other tests)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hold it, i am facing some issues when i have excluded buildJavaSdk task

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, Ping me, when i should rebase

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aman-Mittal Now it should be okay.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamsaghy I have rebased my PRs against your branch you can check PR run on respective PR and
On Push workflow on https://github.com/Aman-Mittal/fineract/actions/workflows/full-build-ci.yml

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Comment thread .github/workflows/build-mysql.yml Outdated
Comment thread .github/workflows/build-progressive-loan.yml
@adamsaghy adamsaghy force-pushed the FINERACT-2421/optimize-github-actions-workflow branch 4 times, most recently from a2c1dca to 416b469 Compare April 23, 2026 17:36
@Aman-Mittal
Copy link
Copy Markdown
Contributor

Aman-Mittal commented Apr 24, 2026 via email

@adamsaghy adamsaghy force-pushed the FINERACT-2421/optimize-github-actions-workflow branch 3 times, most recently from 3c4a544 to ff11734 Compare April 24, 2026 16:31
Comment thread .github/workflows/build-e2e-tests.yml Outdated
@adamsaghy adamsaghy force-pushed the FINERACT-2421/optimize-github-actions-workflow branch from ff11734 to 80db9eb Compare April 25, 2026 00:48
Copy link
Copy Markdown
Contributor Author

@adamsaghy adamsaghy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@Aman-Mittal
Copy link
Copy Markdown
Contributor

@adamsaghy

All failed runs seems to giving cache miss error
enableCrossOsArchive: false
lookup-only: false
env:
DEVELOCITY_ACCESS_KEY:
IMAGE_NAME: fineract
BASE_URL: https://localhost:8443
TEST_USERNAME: mifos
TEST_PASSWORD: password
TEST_STRONG_PASSWORD: A1b2c3d4e5f$
TEST_TENANT_ID: default
INITIALIZATION_ENABLED: true
EVENT_VERIFICATION_ENABLED: true
ACTIVEMQ_BROKER_URL: tcp://localhost:61616
ACTIVEMQ_TOPIC_NAME: events
JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/21.0.11-10/x64
JAVA_HOME_21_X64: /opt/hostedtoolcache/Java_Zulu_jdk/21.0.11-10/x64
Error: Failed to restore cache entry. Exiting as fail-on-cache-miss is set. Input key: fineract-compiled-Linux-24918448689-3

@adamsaghy adamsaghy closed this Apr 25, 2026
@adamsaghy adamsaghy reopened this Apr 25, 2026
Comment thread .github/workflows/build-e2e-tests.yml Outdated
**/build/resources
**/build/tmp
key: fineract-compiled-${{ runner.os }}-${{ github.run_id }}-${{ github.run_attempt }}
fail-on-cache-miss: true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamsaghy Can you reconsider on fail-oncach-miss part - I done a little digging myself it seems that Apache Fineract Repository have only 10 GB of cache limit which can cause cache eviction

Image

@Aman-Mittal
Copy link
Copy Markdown
Contributor

@adamsaghy It seems The caches generated by your CI generating is ranging from 400MB to 900MB each which will likely overload cache of this repo at scale.

@adamsaghy adamsaghy force-pushed the FINERACT-2421/optimize-github-actions-workflow branch from 80db9eb to c216608 Compare April 25, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants