Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

circleci: remove circleci CI tests #3

Closed
wants to merge 23 commits into from
Closed

Conversation

tchaikov
Copy link
Owner

@tchaikov tchaikov commented May 4, 2024

since we are using github CI which enables constributors to run the CI tests when they create a pull request even if they don't have an account on circleci. there is no need to keep circleci anymore.

tchaikov added 13 commits May 3, 2024 00:20
for better readability, so that top-level symbols are more obvious.
as suggested by https://peps.python.org/pep-0008/#blank-lines

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
in future, we only want to print the debugging related on demand.
so, in this change, we extract the helper out() to prepare for the
change.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
prepare the modulization of this script.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
less global variables this way. for better readability.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
less global variables this way. for better readability.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
so it is a local variable now. it's more clear this way. as we can
understand the data dependency better now.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
for better maintainability

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
stall-analyser.py could call `resolve_address()` multiple times
to resolve a certain addresses, but we can avoid feeding `addr2line`
multiple times with the same address by caching the results in memory.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
prepare for the change for adding another render of the backtrace
samplings.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
instead of relying the global variable of `args`, pass
`branch_threshold` to `print_graph()` as a parameter,  it's more
clear this way. as we can understand the data dependency better now.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
this method is not closely related to this class, so let's move
it out. also, by moving it out, we don't need to pass `args` to
`print_graph()` if we don't want to keep `args` as a global
variable. so this change also prepares for the cleanup to
make `args` into `main()` as a local variable. please note,
this changes the order of output, before this change, we print
the command line after the help message explaining how to interpret
the output. after this change, the command line is printed before
the help message.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
so it is not a global variable anymore. for better readability.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
xemul and others added 2 commits May 6, 2024 10:35
…ess() ' from Kefu Chai

this series intends to cleanup stall-analyser by removing more global variables. and cache the results of resolve_address() for better performance.

Closes scylladb#2178

* https://github.com/scylladb/seastar:
  stall-analyser: move `args` into main()
  stall-analyser: move print_command_line_options() out of Graph
  stall-analyser: pass branch_threshold via parameter
  stall-analyser: move process_graph() into Graph class
  scripts: addr2line: cache the results of resolve_address()
  stall-analyser: document the parser of log lines
  stall-analyser: move resolver into main()
  stall-analyser: extract get_command_line_parser() out
  stall-analyser: move graph into main()
  stall-analyser: extract main() out
  stall-analyser: extract print_command_line_options() out
  stall-analyser: add more typing annotatins
  stall-analyser: surround top-level function with two empty lines
before this change, we implement rpc::tuple by inheriting it from
std::tuple, and following the tuple-like protocol, by implementing
`tuple_size<seastar::rpc::tuple<T...>>` and
`tuple_element<I,seastar::rpc::tuple<T...>>`. but in C++23, the
tuple-like constraints are tightened, quote from libstdc++ shipped
along with GCC-14

```c++
  template<typename _Tp>
    inline constexpr bool __is_tuple_v = false;

  template<typename... _Ts>
    inline constexpr bool __is_tuple_v<tuple<_Ts...>> = true;

  // TODO: Reuse __is_tuple_like from <type_traits>?
  template<typename _Tp>
    inline constexpr bool __is_tuple_like_v = false;

  template<typename... _Elements>
    inline constexpr bool __is_tuple_like_v<tuple<_Elements...>> = true;

  template<typename _T1, typename _T2>
    inline constexpr bool __is_tuple_like_v<pair<_T1, _T2>> = true;

  template<typename _Tp, size_t _Nm>
    inline constexpr bool __is_tuple_like_v<array<_Tp, _Nm>> = true;
// ...
```

which is a loyal translation of the C++23 standard.

apparently, `rpc::tuple` cannot fulfill this constraint, hence
`rpc_test.cc` fails to compile with GCC-14 and C++23. fortunately,
`rpc::tuple` is derived from `std::tuple`.

so, in this change, we cast `rpc::tuple` to `std::tuple` when passing
it to `std::apply` to satisfy the more picky critera in C++23.

Fixes scylladb#2158
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov added a commit that referenced this pull request May 7, 2024
in main(), we creates an instance of `http_server_control` using
new, but we never destroy it. this is identified by ASan

```
==2190125==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x55e21cf487bd in operator new(unsigned long) /home/kefu/dev/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:86:3
    #1 0x55e21cf6cf31 in main::$_0::operator()() const::'lambda'()::operator()() const /home/kefu/dev/seastar/apps/httpd/main.cc:121:27
    #2 0x55e21cf6b4cc in int std::__invoke_impl<int, main::$_0::operator()() const::'lambda'()>(std::__invoke_other, main::$_0::operator()() const::'lambda'()&&) /usr/lib/gcc/x86_64-redhat-linux/14/../../../../incl
ude/c++/14/bits/invoke.h:61:14
    #3 0x55e21cf6b46c in std::__invoke_result<main::$_0::operator()() const::'lambda'()>::type std::__invoke<main::$_0::operator()() const::'lambda'()>(main::$_0::operator()() const::'lambda'()&&) /usr/lib/gcc/x86_
64-redhat-linux/14/../../../../include/c++/14/bits/invoke.h:96:14
    scylladb#4 0x55e21cf6b410 in decltype(auto) std::__apply_impl<main::$_0::operator()() const::'lambda'(), std::tuple<>>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&, std::integer_sequence<unsigned long, .
..>) /usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/tuple:2921:14
    scylladb#5 0x55e21cf6b3b2 in decltype(auto) std::apply<main::$_0::operator()() const::'lambda'(), std::tuple<>>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&) /usr/lib/gcc/x86_64-redhat-linux/14/../../../
../include/c++/14/tuple:2936:14
    scylladb#6 0x55e21cf6b283 in seastar::future<int> seastar::futurize<int>::apply<main::$_0::operator()() const::'lambda'()>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&) /home/kefu/dev/seastar/include/sea
star/core/future.hh:2005:28
    scylladb#7 0x55e21cf6b043 in seastar::futurize<std::invoke_result<main::$_0::operator()() const::'lambda'()>::type>::type seastar::async<main::$_0::operator()() const::'lambda'()>(seastar::thread_attributes, main::$_0:
:operator()() const::'lambda'()&&)::'lambda'()::operator()() const /home/kefu/dev/seastar/include/seastar/core/thread.hh:260:13
    scylladb#8 0x55e21cf6ae74 in seastar::noncopyable_function<void ()>::direct_vtable_for<seastar::futurize<std::invoke_result<main::$_0::operator()() const::'lambda'()>::type>::type seastar::async<main::$_0::operator()()
 const::'lambda'()>(seastar::thread_attributes, main::$_0::operator()() const::'lambda'()&&)::'lambda'()>::call(seastar::noncopyable_function<void ()> const*) /home/kefu/dev/seastar/include/seastar/util/noncopyable
_function.hh:129:20
    scylladb#9 0x7f5d757a0fb3 in seastar::noncopyable_function<void ()>::operator()() const /home/kefu/dev/seastar/include/seastar/util/noncopyable_function.hh:215:16
    scylladb#10 0x7f5d75ef5611 in seastar::thread_context::main() /home/kefu/dev/seastar/src/core/thread.cc:311:9
    scylladb#11 0x7f5d75ef50eb in seastar::thread_context::s_main(int, int) /home/kefu/dev/seastar/src/core/thread.cc:287:43
    scylladb#12 0x7f5d72f8a18f  (/lib64/libc.so.6+0x5a18f) (BuildId: b098f1c75a76548bb230d8f551eae07a2aeccf06)
```

so, in this change, let's hold it using a smart pointer, so we
can destroy it when it leaves the lexical scope.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
tchaikov and others added 8 commits May 7, 2024 10:06
in main(), we creates an instance of `http_server_control` using
new, but we never destroy it. this is identified by ASan

```
==2190125==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x55e21cf487bd in operator new(unsigned long) /home/kefu/dev/llvm-project/compiler-rt/lib/asan/asan_new_delete.cpp:86:3
    #1 0x55e21cf6cf31 in main::$_0::operator()() const::'lambda'()::operator()() const /home/kefu/dev/seastar/apps/httpd/main.cc:121:27
    #2 0x55e21cf6b4cc in int std::__invoke_impl<int, main::$_0::operator()() const::'lambda'()>(std::__invoke_other, main::$_0::operator()() const::'lambda'()&&) /usr/lib/gcc/x86_64-redhat-linux/14/../../../../incl
ude/c++/14/bits/invoke.h:61:14
    #3 0x55e21cf6b46c in std::__invoke_result<main::$_0::operator()() const::'lambda'()>::type std::__invoke<main::$_0::operator()() const::'lambda'()>(main::$_0::operator()() const::'lambda'()&&) /usr/lib/gcc/x86_
64-redhat-linux/14/../../../../include/c++/14/bits/invoke.h:96:14
    scylladb#4 0x55e21cf6b410 in decltype(auto) std::__apply_impl<main::$_0::operator()() const::'lambda'(), std::tuple<>>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&, std::integer_sequence<unsigned long, .
..>) /usr/lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/tuple:2921:14
    scylladb#5 0x55e21cf6b3b2 in decltype(auto) std::apply<main::$_0::operator()() const::'lambda'(), std::tuple<>>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&) /usr/lib/gcc/x86_64-redhat-linux/14/../../../
../include/c++/14/tuple:2936:14
    scylladb#6 0x55e21cf6b283 in seastar::future<int> seastar::futurize<int>::apply<main::$_0::operator()() const::'lambda'()>(main::$_0::operator()() const::'lambda'()&&, std::tuple<>&&) /home/kefu/dev/seastar/include/sea
star/core/future.hh:2005:28
    scylladb#7 0x55e21cf6b043 in seastar::futurize<std::invoke_result<main::$_0::operator()() const::'lambda'()>::type>::type seastar::async<main::$_0::operator()() const::'lambda'()>(seastar::thread_attributes, main::$_0:
:operator()() const::'lambda'()&&)::'lambda'()::operator()() const /home/kefu/dev/seastar/include/seastar/core/thread.hh:260:13
    scylladb#8 0x55e21cf6ae74 in seastar::noncopyable_function<void ()>::direct_vtable_for<seastar::futurize<std::invoke_result<main::$_0::operator()() const::'lambda'()>::type>::type seastar::async<main::$_0::operator()()
 const::'lambda'()>(seastar::thread_attributes, main::$_0::operator()() const::'lambda'()&&)::'lambda'()>::call(seastar::noncopyable_function<void ()> const*) /home/kefu/dev/seastar/include/seastar/util/noncopyable
_function.hh:129:20
    scylladb#9 0x7f5d757a0fb3 in seastar::noncopyable_function<void ()>::operator()() const /home/kefu/dev/seastar/include/seastar/util/noncopyable_function.hh:215:16
    scylladb#10 0x7f5d75ef5611 in seastar::thread_context::main() /home/kefu/dev/seastar/src/core/thread.cc:311:9
    scylladb#11 0x7f5d75ef50eb in seastar::thread_context::s_main(int, int) /home/kefu/dev/seastar/src/core/thread.cc:287:43
    scylladb#12 0x7f5d72f8a18f  (/lib64/libc.so.6+0x5a18f) (BuildId: b098f1c75a76548bb230d8f551eae07a2aeccf06)
```

so, in this change, let's hold it using a smart pointer, so we
can destroy it when it leaves the lexical scope.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb#2224
In Seastar, metrics can be defined with implicit aggregation by specific labels,
which occurs at query time. This feature is useful, for instance to define metrics
per shard or even more finely grained per an application-defined entity while reporting
them in a more aggregated manner, such as sum or histogram per server.

However, there are times when it is necessary to inspect the fine-grained metrics.
This can be achieved by adding `__aggregate__=false` to the query string. For example:
`http://localhost:9180/metrics?__aggregate__=false`

Closes scylladb#2219
before this change, contributor is required to sign on CircleCI for
triggering the tests for his/her pull request, but not all contributrs
register this service.

after this change, contributors are able to run tests as long as the
pull request is created on GitHub,

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb#2050
before this change, we

* override the combination of <clang++,23,release>
  by adding settings to build with dpdk enabled,
* override the combination of <clang++,23,debug>
  by adding settings to build with C++20 moduels
  enabled.

but what we intended was to add two additional
builds.

so, in this change, we preserve the existing
combinations of <clang++,23,release> and
<clang++,23,debug> by includeing all the
combinations explicitly, and add the builds
with dpdk and C++20 modules. also the matrix
does not allows one to add extra settings without
overriding an existing combination.
see also https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs

also remove the confusing comment

based on the feedback of reviewer,

> the phrase "only build this combination when dpdk is enable" confusing
> - the phrase "in addition to the above" says the opposite, that we did
> build this combination without dpdk but now we'll build it again with
> dpdk.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb#2227
this change addresses a regression introduced by
3f534c7, which failed to differentiate the path param from
the non-path param, and always prefixed the param with "/".

Fixes 3f534c7
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
before this change, we don't have a test for
scripts/seastar-json2code.py. to verify the behavior of this script,
let's add an integration test which exercise an http server which
uses the source code generated with this script.

Fixes scylladb#2225
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
this change addresses a regression introduced by
3f534c7, which failed to differentiate the path param from the non-path param, and always prefixed the param with "/".

Fixes 3f534c7
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb#2223

* github.com:scylladb/seastar:
  tests/unit: add a unit test for json2code
  seastar-json2code: fix the path param handling
since we are using github CI which enables constributors to run
the CI tests when they create a pull request even if they don't
have an account on circleci. there is no need to keep circleci
anymore.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants