New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offline vulnerabilities check #70

Closed
bluefriday opened this Issue Jul 3, 2018 · 7 comments

Comments

Projects
None yet
5 participants
@bluefriday

bluefriday commented Jul 3, 2018

Hello, I recently used Anchore service and Anchore REST API to check image vulnerabilities.

At first time I ran Anchore for using docker-compose on public GCE and It worked well.

Then I copied 'db' directory to my local pc in order to re-test in my local offline Environment.

But this time, Anchore did not work properly. I could get the manifest / digest value through the API, but the image was not analyzed.

[
    {
        "analysis_status": "not_analyzed",
        "analyzed_at": null,
        "annotations": {},
        "created_at": "2018-07-03T04:33:57Z",
        "imageDigest": "sha256:a08ed346dfbb55cf7819dbe60f574f19fe387f2e7486cdc2073f1272d1344ec9",
        "image_content": {
            "metadata": {
                "arch": null,
                "distro": null,
                "distro_version": null,
                "dockerfile_mode": null,
                "image_size": null,
                "layer_count": null
            }
        },
...

I would like to test the Anchore service in offline environment.
In this case, what else do I need to do In addition to moving the db directory?

@nurmi

This comment has been minimized.

Show comment
Hide comment
@nurmi

nurmi Jul 3, 2018

Member

hi @bluefriday - I think we'll need a little bit more information, but we can start by sharing some context about the meaning of this state.

When an image is first 'added' to anchore-engine, its initial state will be 'not_analyzed' (which is an OK state for an image to be in). If there is an analyzer service that is successfully up and running, and the rest of the anchore services are also up and running, then the analyzer will wake up and see if there are any new images in queue to be analyzed (i.e. in not_analyzed state), and it should then pull the job and move the image to 'analyzing' state. From there, the image will either be successfully analyzed (moving it to 'analyzed' state) or if something goes wrong, will put the image in 'analysis_failed' state).

WIth that being said - you should always see an image enter the 'not_analyzed' state when it is first added, but if all is going well, you should see it next go to 'analyzing', if you were to query the image (say, using anchore-cli image get <image> to watch the image move along. If it is not moving along, then you should review the service status of the engine and determine why the image is not being picked up for analysis - anchore-cli system status should give us some insight.

If an analyzer is not up and running, that would be the situation to try and diagnose (you should see startup failures when you bring up the service in docker logs <anchore engine container> or review the contents of /var/log/anchore/anchore-worker.log inside the engine container that is running your analyzer.

Best
-Dan

Member

nurmi commented Jul 3, 2018

hi @bluefriday - I think we'll need a little bit more information, but we can start by sharing some context about the meaning of this state.

When an image is first 'added' to anchore-engine, its initial state will be 'not_analyzed' (which is an OK state for an image to be in). If there is an analyzer service that is successfully up and running, and the rest of the anchore services are also up and running, then the analyzer will wake up and see if there are any new images in queue to be analyzed (i.e. in not_analyzed state), and it should then pull the job and move the image to 'analyzing' state. From there, the image will either be successfully analyzed (moving it to 'analyzed' state) or if something goes wrong, will put the image in 'analysis_failed' state).

WIth that being said - you should always see an image enter the 'not_analyzed' state when it is first added, but if all is going well, you should see it next go to 'analyzing', if you were to query the image (say, using anchore-cli image get <image> to watch the image move along. If it is not moving along, then you should review the service status of the engine and determine why the image is not being picked up for analysis - anchore-cli system status should give us some insight.

If an analyzer is not up and running, that would be the situation to try and diagnose (you should see startup failures when you bring up the service in docker logs <anchore engine container> or review the contents of /var/log/anchore/anchore-worker.log inside the engine container that is running your analyzer.

Best
-Dan

@bluefriday

This comment has been minimized.

Show comment
Hide comment
@bluefriday

bluefriday Jul 4, 2018

@nurmi Thanks for your answer.

I took your advice and analyzed the anchore log.
After checking the five modules (api, catalog, k8s, simpleQ, worker) in the "/var/log" path, the log except for catalog was normal.

Below is the log in the catalog.

...
Site starting on 8082
Starting factory <twisted.web.server.Site instance at 0x7f529d6c0c20>
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.services.catalog/handle_policy_bundle_sync()] [INFO] policy engine is not yet ready - retrying 0/10
[Thread-16] [anchore_engine.services.catalog/handle_service_watcher()] [WARN] no service heartbeat within max allowed time period ([u'dockerhostid-anchore-engine', u'http://anchore-engine:8087'] - orphaning service
[Thread-14] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-18] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-20] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.services.catalog/handle_policy_bundle_sync()] [INFO] policy engine is not yet ready - retrying 1/10
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
...

The api request (/v1/system/services) for the engine is also shown below.

...
},
{
	"base_url": "http://anchore-engine:8087",
	"hostid": "dockerhostid-anchore-engine",
	"service_detail": "no heartbeat from service in (3600) seconds",
	"servicename": "policy_engine",
	"status": false,
	"status_message": "orphaned",
	"version": "v1"
},
{
...

I think this problem occurs because the policy-engine is not running properly.
However, we could not find any special part in the policy-engine configuration.

  policy_engine:
    enabled: True
    require_auth: True
    endpoint_hostname: '${ANCHORE_HOST_ID}'
    listen: '0.0.0.0'
    port: 8087
#    external_port: 8087
#    external_tls: False
    cycle_timer_seconds: 1
    cycle_timers:
      feed_sync: 21600 # 6 hours between feed syncs
      feed_sync_checker: 3600 # 1 hour between checks to see if there needs to be a task queue

I would like to ask if there are any additional settings I can set.

bluefriday commented Jul 4, 2018

@nurmi Thanks for your answer.

I took your advice and analyzed the anchore log.
After checking the five modules (api, catalog, k8s, simpleQ, worker) in the "/var/log" path, the log except for catalog was normal.

Below is the log in the catalog.

...
Site starting on 8082
Starting factory <twisted.web.server.Site instance at 0x7f529d6c0c20>
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.services.catalog/handle_policy_bundle_sync()] [INFO] policy engine is not yet ready - retrying 0/10
[Thread-16] [anchore_engine.services.catalog/handle_service_watcher()] [WARN] no service heartbeat within max allowed time period ([u'dockerhostid-anchore-engine', u'http://anchore-engine:8087'] - orphaning service
[Thread-14] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-18] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-20] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
[Thread-11] [anchore_engine.services.catalog/handle_policy_bundle_sync()] [INFO] policy engine is not yet ready - retrying 1/10
[Thread-11] [anchore_engine.clients.common/check_services_ready()] [WARN] required service (policy_engine) is not (yet) available
...

The api request (/v1/system/services) for the engine is also shown below.

...
},
{
	"base_url": "http://anchore-engine:8087",
	"hostid": "dockerhostid-anchore-engine",
	"service_detail": "no heartbeat from service in (3600) seconds",
	"servicename": "policy_engine",
	"status": false,
	"status_message": "orphaned",
	"version": "v1"
},
{
...

I think this problem occurs because the policy-engine is not running properly.
However, we could not find any special part in the policy-engine configuration.

  policy_engine:
    enabled: True
    require_auth: True
    endpoint_hostname: '${ANCHORE_HOST_ID}'
    listen: '0.0.0.0'
    port: 8087
#    external_port: 8087
#    external_tls: False
    cycle_timer_seconds: 1
    cycle_timers:
      feed_sync: 21600 # 6 hours between feed syncs
      feed_sync_checker: 3600 # 1 hour between checks to see if there needs to be a task queue

I would like to ask if there are any additional settings I can set.

@zhill

This comment has been minimized.

Show comment
Hide comment
@zhill

zhill Jul 5, 2018

Member

Hi @bluefriday, the Anchore Engine currently has an issue that prevents it from running properly in a fully offline mode where it has no network internet access. The issue is that the policy engine component expects to be able to connect to an external feed service endpoint (by default https://ancho.re) in order to sync vulnerability data to keep image analyses up-to-date with the latest cve sources. There is another issue, #54, that is a request for this same feature: disable the feed sync cleanly in the config.yaml to enable the policy engine to run without any feed updates. For now, progress on the enhancement to disable the feed-sync will be primarily done in #54, so watch that issue. We're planning on addressing it soon, but currently do not have a specific ETA for that change.

I should mention that our Enterprise version of Anchore Engine does have a local feed service that you configure the policy-engine to connect to so with Enterprise it is possible to run in a fully offline mode. If you're interested in trying that let me know and we can discuss further.

Member

zhill commented Jul 5, 2018

Hi @bluefriday, the Anchore Engine currently has an issue that prevents it from running properly in a fully offline mode where it has no network internet access. The issue is that the policy engine component expects to be able to connect to an external feed service endpoint (by default https://ancho.re) in order to sync vulnerability data to keep image analyses up-to-date with the latest cve sources. There is another issue, #54, that is a request for this same feature: disable the feed sync cleanly in the config.yaml to enable the policy engine to run without any feed updates. For now, progress on the enhancement to disable the feed-sync will be primarily done in #54, so watch that issue. We're planning on addressing it soon, but currently do not have a specific ETA for that change.

I should mention that our Enterprise version of Anchore Engine does have a local feed service that you configure the policy-engine to connect to so with Enterprise it is possible to run in a fully offline mode. If you're interested in trying that let me know and we can discuss further.

@KoreaSecurity

This comment has been minimized.

Show comment
Hide comment
@KoreaSecurity

KoreaSecurity Jul 7, 2018

It seems to have blocked IP of some countries.
Try connecting to another country's IP.

KoreaSecurity commented Jul 7, 2018

It seems to have blocked IP of some countries.
Try connecting to another country's IP.

@nurmi

This comment has been minimized.

Show comment
Hide comment
@nurmi

nurmi Jul 18, 2018

Member

update on the ability to disable feed syncs in anchore-engine - we've added a new configuration parameter that allows feed syncs to be disabled in commit 'd61f643d3652579d105b4500e6da6bf804cbf073', which is also available in the latest unstable container build (docker.io/anchore/anchore-engine:dev), for from-scratch testing purposes.

The new parameter is called 'sync_enabled: <True|False>', inside the 'feeds' section of config.yaml.

That functionality should allow for the engine to come up, in the use case described in the original report!

Member

nurmi commented Jul 18, 2018

update on the ability to disable feed syncs in anchore-engine - we've added a new configuration parameter that allows feed syncs to be disabled in commit 'd61f643d3652579d105b4500e6da6bf804cbf073', which is also available in the latest unstable container build (docker.io/anchore/anchore-engine:dev), for from-scratch testing purposes.

The new parameter is called 'sync_enabled: <True|False>', inside the 'feeds' section of config.yaml.

That functionality should allow for the engine to come up, in the use case described in the original report!

@stefanvangastel

This comment has been minimized.

Show comment
Hide comment
@stefanvangastel

stefanvangastel Jul 18, 2018

@nurmi Thanks for pointing me here, is there any other info / docs on air gap installation and usage. Or could I contact someone for more info?

stefanvangastel commented Jul 18, 2018

@nurmi Thanks for pointing me here, is there any other info / docs on air gap installation and usage. Or could I contact someone for more info?

@nurmi

This comment has been minimized.

Show comment
Hide comment
@nurmi

nurmi Jul 18, 2018

Member

@stefanvangastel - We do have solutions for completely air-gapped deployments (best achieved with our Enterprise on-prem feed service) - probably best to contact to contact us directly (nurmi@anchore.com) and/or join our slack (https://anchore.com/slack) so that we can discuss options

Member

nurmi commented Jul 18, 2018

@stefanvangastel - We do have solutions for completely air-gapped deployments (best achieved with our Enterprise on-prem feed service) - probably best to contact to contact us directly (nurmi@anchore.com) and/or join our slack (https://anchore.com/slack) so that we can discuss options

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment