Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Sync with ostree #1000

Closed
wants to merge 3 commits into from
Closed

ci: Sync with ostree #1000

wants to merge 3 commits into from

Conversation

cgwalters
Copy link
Member

We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

@cgwalters
Copy link
Member Author

bot, retest this please

@jlebon
Copy link
Member

jlebon commented Sep 19, 2017

Seems like the error is:

gcc: error: unrecognized command line option '-fsanitize=undefined'
gcc: error: unrecognized command line option '-fsanitize-undefined-trap-on-error'

@cgwalters
Copy link
Member Author

Yeah, but it's weird; it should be the same setup as ostree right?

@cgwalters
Copy link
Member Author

bot, retest this please

1 similar comment
@cgwalters
Copy link
Member Author

bot, retest this please

@jlebon
Copy link
Member

jlebon commented Sep 19, 2017

@rh-atomic-bot r+ d215984

@cgwalters
Copy link
Member Author

@rh-atomic-bot r=jlebon 1edea17

rh-atomic-bot pushed a commit that referenced this pull request Sep 19, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

Closes: #1000
Approved by: jlebon
@rh-atomic-bot
Copy link

⌛ Testing commit 1edea17 with merge 97147e2...

@rh-atomic-bot
Copy link

💔 Test failed - status-atomicjenkins

@jlebon
Copy link
Member

jlebon commented Sep 20, 2017

@rh-atomic-bot retry

@rh-atomic-bot
Copy link

⌛ Testing commit 1edea17 with merge 278591d...

rh-atomic-bot pushed a commit that referenced this pull request Sep 20, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

Closes: #1000
Approved by: jlebon
@jlebon
Copy link
Member

jlebon commented Sep 20, 2017

@rh-atomic-bot retry

@rh-atomic-bot
Copy link

⌛ Testing commit 1edea17 with merge ee832bb...

rh-atomic-bot pushed a commit that referenced this pull request Sep 20, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

Closes: #1000
Approved by: jlebon
@jlebon
Copy link
Member

jlebon commented Sep 20, 2017

Just noticed the PR# on this one. 🍾

@cgwalters
Copy link
Member Author

Indeed! 🎊

@rh-atomic-bot
Copy link

💥 Test timed out

@jlebon
Copy link
Member

jlebon commented Sep 21, 2017

@rh-atomic-bot retry

rh-atomic-bot pushed a commit that referenced this pull request Sep 21, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

Closes: #1000
Approved by: jlebon
@rh-atomic-bot
Copy link

⌛ Testing commit 1edea17 with merge 2773af6...

@rh-atomic-bot
Copy link

💔 Test failed - status-atomicjenkins

@jlebon
Copy link
Member

jlebon commented Sep 21, 2017

Hmm, both times it failed at the exact same spot:

PASS: tests/check/jsonutil 1 /jsonparsing/get-optional-member
PASS: tests/check/jsonutil 2 /versioning/automatic
PASS: tests/check/postprocess 1 /altfiles
PASS: tests/check/test-utils 1 /utils/varsubst
PASS: tests/check/test-utils 2 /utils/cachebranch_to_nevra
PASS: tests/check/test-utils 3 /unpacker/variant_to_nevra
### TIMED OUT AFTER 3581s

I'm starting to think this might not be a flake. I wonder if one of the new flags is causing a hang somehow.

@jlebon
Copy link
Member

jlebon commented Sep 21, 2017

Odd, it works fine locally at the very least. Let's give it one more try.
@rh-atomic-bot retry

@rh-atomic-bot
Copy link

⌛ Testing commit 1edea17 with merge 436809e...

rh-atomic-bot pushed a commit that referenced this pull request Sep 21, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

Closes: #1000
Approved by: jlebon
@rh-atomic-bot
Copy link

💔 Test failed - status-atomicjenkins

@cgwalters
Copy link
Member Author

cgwalters commented Sep 21, 2017

Whoo, so this is fun. First off, I got a stack from at least one hang:

deadlock with asan + gpgme
Thread 1 (Thread 0x7f8fdcbe1980 (LWP 2283)):
#0  0x00007f8fdbbc6bee in  () at /lib64/libasan.so.4
#1  0x00007f8fdbaf46e1 in  () at /lib64/libasan.so.4
#2  0x00007f8fdbaf485e in  () at /lib64/libasan.so.4
#3  0x00007f8fdbaf174c in  () at /lib64/libasan.so.4
#4  0x00007f8fdbba781b in malloc () at /lib64/libasan.so.4
#5  0x00007f8fd8328482 in opendir () at /lib64/libc.so.6
#6  0x00007f8fdbb0a71c in  () at /lib64/libasan.so.4
#7  0x00007f8fdaed601b in get_max_fds () at posix-io.c:300
#8  0x00007f8fdaed601b in _gpgme_io_spawn (path=path@entry=0x60200025c8f0 "/usr/bin/gpg2", argv=0x60e000003300, flags=flags@entry=3, fd_list=fd_list@entry=0x607000012070, atfork=atfork@entry=0x0, atforkvalue=atforkvalue@entry=0x0, r_pid=0x7ffc07ebf838) at posix-io.c:456
#9  0x00007f8fdaec8a97 in start (gpg=0x613000020f00) at engine-gpg.c:1470
#10 0x00007f8fdaeb99e7 in gpgme_op_verify (ctx=0x612000035140, sig=0x621000038500, signed_text=0x621000037100, plaintext=0x0) at verify.c:1146
#11 0x00007f8fdb15fa4c in _ostree_gpg_verifier_check_signature (self=0x619000028240, signed_data=0x6190000a4240, signatures=0x61900004c660, cancellable=0x0, error=0x7ffc07ebfcd0)
    at src/libostree/ostree-gpg-verifier.c:229
#12 0x00007f8fdb118581 in _ostree_repo_gpg_verify_data_internal (self=0x625000008000, remote_name=0x60200025c630 "secureos", data=0x6190000a4240, signatures=0x61900004c660, keyringdir=0x0, extra_keyring=0x0, cancellable=0x0, error=0x7ffc07ebfcd0) at src/libostree/ostree-repo.c:4327
#13 0x00007f8fdb118710 in _ostree_repo_gpg_verify_with_metadata (self=0x625000008000, signed_data=0x6190000a4240, metadata=0x6190000218d0, remote_name=0x60200025c630 "secureos", keyringdir=0x0, extra_keyring=0x0, cancellable=0x0, error=0x7ffc07ebfcd0) at src/libostree/ostree-repo.c:4380
#14 0x00007f8fdb11883c in _ostree_repo_verify_commit_internal (self=0x625000008000, commit_checksum=0x607000011d60 "446736aa378317053221305531df1317b8045af36a865967ff58cefbe86d13d3", remote_name=0x60200025c630 "secureos", keyringdir=0x0, extra_keyring=0x0, cancellable=0x0, error=0x7ffc07ebfcd0) at src/libostree/ostree-repo.c:4424
#15 0x00007f8fdb1189b4 in ostree_repo_verify_commit_for_remote (self=0x625000008000, commit_checksum=0x607000011d60 "446736aa378317053221305531df1317b8045af36a865967ff58cefbe86d13d3", remote_name=0x60200025c630 "secureos", cancellable=0x0, error=0x7ffc07ebfcd0) at src/libostree/ostree-repo.c:4515
#16 0x00000000004ada00 in rpmostreed_deployment_gpg_results ()
#17 0x00000000004af1b5 in rpmostreed_deployment_generate_variant ()
#18 0x00000000004aa741 in sysroot_populate_deployments_unlocked ()
#19 0x00000000004acc3a in rpmostreed_sysroot_reload ()
#20 0x00000000004ad060 in on_deploy_changed ()
#21 0x00007f8fd1e59bde in ffi_call_unix64 () at /lib64/libffi.so.6
#22 0x00007f8fd1e5954f in ffi_call () at /lib64/libffi.so.6
#23 0x00007f8fd8d70efd in g_cclosure_marshal_generic_va () at /lib64/libgobject-2.0.so.0
#24 0x00007f8fd8d70546 in _g_closure_invoke_va () at /lib64/libgobject-2.0.so.0
#25 0x00007f8fd8d8b494 in g_signal_emit_valist () at /lib64/libgobject-2.0.so.0
#26 0x00007f8fd8d8bb0f in g_signal_emit () at /lib64/libgobject-2.0.so.0
#27 0x00007f8fd90974a9 in g_file_monitor_source_dispatch () at /lib64/libgio-2.0.so.0
#28 0x00007f8fd8a98247 in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#29 0x00007f8fd8a985e8 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
#30 0x00007f8fd8a9867c in g_main_context_iteration () at /lib64/libglib-2.0.so.0
#31 0x00000000004a5bfc in rpmostreed_daemon_run_until_idle_exit ()
#32 0x000000000043f8e0 in rpmostree_builtin_start_daemon ()
#33 0x000000000041bdf6 in main ()

And I think somehow ASAN is badly provoking this bug:

https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpgme.git;a=commit;h=93a59070c699d569d1eac7ba22355fe3f5d10882

Probably ASAN is trying to hook fork(). I'm not entirely sure why we're not seeing it with ostree right now.

This is the rpm-ostree equivalent of
<ostreedev/ostree@47b4dd1>

Unfortunately, introspection uses `dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Prep for syncing PAPR config with ostree.
cgwalters added a commit to cgwalters/rpm-ostree that referenced this pull request Sep 21, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<coreos#1000 (comment)>
@cgwalters
Copy link
Member Author

OK, let's try this...

We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<coreos#1000 (comment)>
# might get passed to it.
id=$(. /etc/os-release && echo $ID)
if [ "$id" == fedora ]; then
git clean -dfx
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out that libdnf is not ready to build with clang, and we'd subtly been avoiding that by not doing a git submodule foreach git clean -dfx like ostree does. For now let's just back this out while I try to unwind that.

@jlebon
Copy link
Member

jlebon commented Sep 24, 2017

@rh-atomic-bot r+ 41e520d

@rh-atomic-bot
Copy link

⌛ Testing commit 41e520d with merge 2ebf43f...

rh-atomic-bot pushed a commit that referenced this pull request Sep 24, 2017
We have some drift; keep this more in sync so we maintain the rpm/yum
hackarounds for example in one place.

I backed out the ASAN bits though pending at least a newer gpgme:
<#1000 (comment)>

Closes: #1000
Approved by: jlebon
@rh-atomic-bot
Copy link

☀️ Test successful - status-atomicjenkins
Approved by: jlebon
Pushing 2ebf43f to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants