Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #2900 add write-config flag and cluster start #2903

Merged
merged 1 commit into from Dec 12, 2018

Conversation

praveenkumar
Copy link
Contributor

@praveenkumar praveenkumar commented Oct 17, 2018

Fixes #2900

Steps to test the pull request

$ export MINISHIFT_ENABLE_EXPERIMENTAL=y
$ minishift start --write-config
$ minishift openshift start

@praveenkumar praveenkumar force-pushed the issue_2900 branch 3 times, most recently from b05da55 to 37196ce Compare October 17, 2018 13:41
@praveenkumar praveenkumar changed the title [WIP] Issue #2900 add write-config flag and cluster start Issue #2900 add write-config flag and cluster start Oct 17, 2018
@LalatenduMohanty
Copy link
Member

@praveenkumar In terms of directories in /var/lib/minishift/base I think I see the same number of directories. Can you please point me the config file that gets created with --write-config?

@LalatenduMohanty
Copy link
Member

@praveenkumar Also wondering what is the downside of enabling --write-config in the clusterup.ClusterUp() by default.

@praveenkumar
Copy link
Contributor Author

Can you please point me the config file that gets created with --write-config?

@LalatenduMohanty it will not create any other directory but this is the flag to create the config directory without initiating the cluster up.

@LalatenduMohanty
Copy link
Member

CI failure is not related to this PR.

--- Failed steps:
08:44:22 
08:44:22   Scenario: User deploys nodejs example application from OpenShift repository # features/cmd-openshift.feature:50
08:44:22     Then stdout should contain # features/cmd-openshift.feature:53
08:44:22       Error: Output did not match. Expected: 'Run 'oc status' to view your app.', Actual: '--> Found image 93de123 (4 days old) in image stream "openshift/nodejs" under tag "10" for "nodejs"
08:44:22 
08:44:22     Node.js 10.12.0 
08:44:22     --------------- 
08:44:22     Node.js  available as docker container is a base platform for building and running various Node.js  applications and frameworks. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.
08:44:22 
08:44:22     Tags: builder, nodejs, nodejs-10.12.0
08:44:22 
08:44:22     * The source repository appears to match: nodejs
08:44:22     * A source build using source code from https://github.com/sclorg/nodejs-ex will be created
08:44:22       * The resulting image will be pushed to image stream tag "nodejs-ex:latest"
08:44:22       * Use 'start-build' to trigger a new build
08:44:22     * This image will be deployed in deployment config "nodejs-ex"
08:44:22     * Port 8080/tcp will be load balanced by service "nodejs-ex"
08:44:22       * Other containers can access this service through the hostname "nodejs-ex"
08:44:22 
08:44:22 --> Creating resources with label name=myapp ...
08:44:22 --> Failed

https://ci.centos.org/job/minishift-pr/5261/consoleFull

@LalatenduMohanty
Copy link
Member

@praveenkumar This PR is failing for me consistently , can you take a look?


$ minishift start --write-config
xxxxxxxxxxxxxxxxxxxxxx
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I1106 08:53:38.645232    2099 config.go:40] Running "create-master-config"
I1106 08:53:42.487112    2099 config.go:46] Running "create-node-config"
Wrote config to: "/var/lib/minishift/base"

$  minishift openshift start
...............................................................Error during 'cluster up' execution: Error starting the cluster. ssh command error:
command : /var/lib/minishift/bin/oc cluster up --public-hostname 192.168.42.211 --routing-suffix 192.168.42.211.nip.io --base-dir /var/lib/minishift/base --image 'openshift/origin-${component}:v3.11.0'
err     : exit status 2
output  : Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP 192.168.42.211 as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I1106 09:01:16.610581    2560 flags.go:30] Running "create-kubelet-flags"
I1106 09:01:17.273451    2560 run_kubelet.go:49] Running "start-kubelet"
I1106 09:01:17.498499    2560 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I1106 09:02:33.562948    2560 interface.go:26] Installing "kube-proxy" ...
I1106 09:02:33.563705    2560 interface.go:26] Installing "kube-dns" ...
I1106 09:02:33.563714    2560 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I1106 09:02:33.563721    2560 interface.go:26] Installing "openshift-apiserver" ...
I1106 09:02:33.563754    2560 apply_template.go:81] Installing "openshift-apiserver"
I1106 09:02:33.564335    2560 apply_template.go:81] Installing "kube-dns"
I1106 09:02:33.565275    2560 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I1106 09:02:33.566457    2560 apply_template.go:81] Installing "kube-proxy"
I1106 09:02:40.255643    2560 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I1106 09:05:11.408178    2560 run_self_hosted.go:242] openshift-apiserver available
I1106 09:05:11.409033    2560 interface.go:26] Installing "openshift-controller-manager" ...
I1106 09:05:11.409058    2560 apply_template.go:81] Installing "openshift-controller-manager"
I1106 09:05:15.117393    2560 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding web-console ...
Adding centos-imagestreams ...
Adding persistent-volumes ...
Adding registry ...
Adding router ...
Adding sample-templates ...
I1106 09:05:15.334917    2560 interface.go:26] Installing "openshift-web-console-operator" ...
I1106 09:05:15.334938    2560 interface.go:26] Installing "centos-imagestreams" ...
I1106 09:05:15.334951    2560 interface.go:26] Installing "persistent-volumes" ...
I1106 09:05:15.334963    2560 interface.go:26] Installing "openshift-image-registry" ...
I1106 09:05:15.334977    2560 interface.go:26] Installing "openshift-router" ...
I1106 09:05:15.334998    2560 interface.go:26] Installing "sample-templates" ...
I1106 09:05:15.338363    2560 apply_template.go:81] Installing "openshift-web-console-operator"
I1106 09:05:15.351805    2560 interface.go:26] Installing "sample-templates/mysql" ...
I1106 09:05:15.351836    2560 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I1106 09:05:15.351864    2560 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I1106 09:05:15.351874    2560 interface.go:26] Installing "sample-templates/django quickstart" ...
I1106 09:05:15.351882    2560 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I1106 09:05:15.351889    2560 interface.go:26] Installing "sample-templates/rails quickstart" ...
I1106 09:05:15.351896    2560 interface.go:26] Installing "sample-templates/sample pipeline" ...
I1106 09:05:15.351902    2560 interface.go:26] Installing "sample-templates/mariadb" ...
I1106 09:05:15.351911    2560 interface.go:26] Installing "sample-templates/postgresql" ...
I1106 09:05:15.351918    2560 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I1106 09:05:15.351924    2560 interface.go:26] Installing "sample-templates/mongodb" ...
I1106 09:05:15.352006    2560 apply_list.go:67] Installing "sample-templates/mongodb"
I1106 09:05:15.352921    2560 apply_list.go:67] Installing "sample-templates/mysql"
I1106 09:05:15.353106    2560 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I1106 09:05:15.353266    2560 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I1106 09:05:15.353394    2560 apply_list.go:67] Installing "sample-templates/django quickstart"
I1106 09:05:15.353530    2560 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I1106 09:05:15.353655    2560 apply_list.go:67] Installing "sample-templates/rails quickstart"
I1106 09:05:15.353773    2560 apply_list.go:67] Installing "sample-templates/sample pipeline"
I1106 09:05:15.353887    2560 apply_list.go:67] Installing "sample-templates/mariadb"
I1106 09:05:15.354005    2560 apply_list.go:67] Installing "sample-templates/postgresql"
I1106 09:05:15.354112    2560 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I1106 09:05:15.354428    2560 apply_list.go:67] Installing "centos-imagestreams"
E1106 09:05:55.373556    2560 interface.go:34] Failed to install "openshift-router": could not run "install-router": rc=1
panic: Get https://192.168.42.211:8443/apis/apiregistration.k8s.io/v1beta1/apiservices?watch=true: dial tcp 192.168.42.211:8443: connect: connection refused

goroutine 1077 [running]:
github.com/openshift/origin/pkg/oc/clusterup.watchAPIServices(0x30bbda0, 0xc4204b32a0)
	/tmp/openshift/build-rpms/rpm/BUILD/origin-3.11.0/_output/local/go/src/github.com/openshift/origin/pkg/oc/clusterup/run_self_hosted.go:591 +0x5f6
created by github.com/openshift/origin/pkg/oc/clusterup.(*ClusterUpConfig).StartSelfHosted
	/tmp/openshift/build-rpms/rpm/BUILD/origin-3.11.0/_output/local/go/src/github.com/openshift/origin/pkg/oc/clusterup/run_self_hosted.go:244 +0xdad


clusterUpFlagSet = cmdUtil.InitClusterUpFlags("start")
startCmd.Flags().AddFlagSet(clusterUpFlagSet)
OpenShiftCmd.AddCommand(startCmd)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@praveenkumar I found during my testing that minishift openshift start is not behind the experimental flag and also did not find any code for the same.

@praveenkumar
Copy link
Contributor Author

This PR is failing for me consistently , can you take a look?

@LalatenduMohanty I tried on clean environment and it works as expected (xhyve), @amitkrout @agajdosi any one of you can try it and let me know if you are facing similar issue?

$ ./minishift start --write-config
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://github.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.11.0' is valid ... OK
-- Checking if requested OpenShift version 'v3.11.0' is supported ... OK
-- Checking if requested hypervisor 'xhyve' is supported on this platform ... OK
-- Checking if xhyve driver is installed ... 
   Driver is available at /usr/local/bin/docker-machine-driver-xhyve
   Checking for setuid bit ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting the OpenShift cluster using 'xhyve' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Starting Minishift VM ...................... OK
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ... 
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ... 
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- Writing current configuration for static assignment of IP address ... OK
   Importing 'docker.io/openshift/origin-cli:v3.11.0' ........ OK
   Importing 'docker.io/openshift/origin-control-plane:v3.11.0' ............. OK
   Importing 'docker.io/openshift/origin-deployer:v3.11.0' .... OK
   Importing 'docker.io/openshift/origin-docker-registry:v3.11.0' .... OK
   Importing 'docker.io/openshift/origin-haproxy-router:v3.11.0' .... OK
   Importing 'docker.io/openshift/origin-hyperkube:v3.11.0' ...... OK
   Importing 'docker.io/openshift/origin-hypershift:v3.11.0' ......... OK
   Importing 'docker.io/openshift/origin-node:v3.11.0' ................ OK
   Importing 'docker.io/openshift/origin-pod:v3.11.0' .... OK
   Importing 'docker.io/openshift/origin-service-serving-cert-signer:v3.11' ... OK
   Importing 'docker.io/openshift/origin-web-console:v3.11.0' ..... OK
   Importing 'openshift/origin-control-plane:v3.11.0' ........... OK
   Importing 'openshift/origin-docker-registry:v3.11.0' ... OK
   Importing 'openshift/origin-haproxy-router:v3.11.0' ..... OK
-- OpenShift cluster will be configured with ...
   Version: v3.11.0
-- Copying oc binary from the OpenShift container image to VM ... OK
-- Starting OpenShift cluster ...
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP 192.168.64.74 as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I1112 08:33:54.984154    2386 config.go:40] Running "create-master-config"
I1112 08:33:57.525427    2386 config.go:46] Running "create-node-config"
Wrote config to: "/var/lib/minishift/base"


$ ./minishift openshift start
..................................................................................
Getting a Docker client ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Checking type of volume mount ...
Determining server IP ...
Using public hostname IP 192.168.64.74 as the host IP
Checking if OpenShift is already running ...
Checking for supported Docker version (=>1.22) ...
Checking if insecured registry is configured properly in Docker ...
Checking if required ports are available ...
Checking if OpenShift client is configured properly ...
Checking if image openshift/origin-control-plane:v3.11.0 is available ...
Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...
I1112 08:38:12.482161    2788 flags.go:30] Running "create-kubelet-flags"
I1112 08:38:12.878182    2788 run_kubelet.go:49] Running "start-kubelet"
I1112 08:38:13.071623    2788 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I1112 08:42:09.118205    2788 interface.go:26] Installing "kube-proxy" ...
I1112 08:42:09.121060    2788 interface.go:26] Installing "kube-dns" ...
I1112 08:42:09.121102    2788 interface.go:26] Installing "openshift-service-cert-signer-operator" ...
I1112 08:42:09.121127    2788 interface.go:26] Installing "openshift-apiserver" ...
I1112 08:42:09.121972    2788 apply_template.go:81] Installing "openshift-apiserver"
I1112 08:42:09.122643    2788 apply_template.go:81] Installing "kube-proxy"
I1112 08:42:09.124186    2788 apply_template.go:81] Installing "kube-dns"
I1112 08:42:09.125470    2788 apply_template.go:81] Installing "openshift-service-cert-signer-operator"
I1112 08:42:21.793082    2788 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operator" "openshift-apiserver"
I1112 08:44:45.872031    2788 run_self_hosted.go:242] openshift-apiserver available
I1112 08:44:45.873271    2788 interface.go:26] Installing "openshift-controller-manager" ...
I1112 08:44:45.873389    2788 apply_template.go:81] Installing "openshift-controller-manager"
I1112 08:44:52.552233    2788 interface.go:41] Finished installing "openshift-controller-manager"
Adding default OAuthClient redirect URIs ...
Adding persistent-volumes ...
Adding web-console ...
Adding router ...
Adding registry ...
Adding sample-templates ...
Adding centos-imagestreams ...
I1112 08:44:52.615260    2788 interface.go:26] Installing "persistent-volumes" ...
I1112 08:44:52.615310    2788 interface.go:26] Installing "openshift-web-console-operator" ...
I1112 08:44:52.615338    2788 interface.go:26] Installing "openshift-router" ...
I1112 08:44:52.615363    2788 interface.go:26] Installing "openshift-image-registry" ...
I1112 08:44:52.615389    2788 interface.go:26] Installing "sample-templates" ...
I1112 08:44:52.615436    2788 interface.go:26] Installing "centos-imagestreams" ...
I1112 08:44:52.615607    2788 apply_list.go:67] Installing "centos-imagestreams"
I1112 08:44:52.617071    2788 apply_template.go:81] Installing "openshift-web-console-operator"
I1112 08:44:52.618462    2788 interface.go:26] Installing "sample-templates/mariadb" ...
I1112 08:44:52.618495    2788 interface.go:26] Installing "sample-templates/dancer quickstart" ...
I1112 08:44:52.618521    2788 interface.go:26] Installing "sample-templates/django quickstart" ...
I1112 08:44:52.618547    2788 interface.go:26] Installing "sample-templates/rails quickstart" ...
I1112 08:44:52.618572    2788 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ...
I1112 08:44:52.618598    2788 interface.go:26] Installing "sample-templates/sample pipeline" ...
I1112 08:44:52.618623    2788 interface.go:26] Installing "sample-templates/mongodb" ...
I1112 08:44:52.618647    2788 interface.go:26] Installing "sample-templates/mysql" ...
I1112 08:44:52.618674    2788 interface.go:26] Installing "sample-templates/postgresql" ...
I1112 08:44:52.618699    2788 interface.go:26] Installing "sample-templates/cakephp quickstart" ...
I1112 08:44:52.618724    2788 interface.go:26] Installing "sample-templates/nodejs quickstart" ...
I1112 08:44:52.618811    2788 apply_list.go:67] Installing "sample-templates/nodejs quickstart"
I1112 08:44:52.619226    2788 apply_list.go:67] Installing "sample-templates/mariadb"
I1112 08:44:52.619461    2788 apply_list.go:67] Installing "sample-templates/dancer quickstart"
I1112 08:44:52.619702    2788 apply_list.go:67] Installing "sample-templates/django quickstart"
I1112 08:44:52.619987    2788 apply_list.go:67] Installing "sample-templates/rails quickstart"
I1112 08:44:52.620249    2788 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral"
I1112 08:44:52.620474    2788 apply_list.go:67] Installing "sample-templates/sample pipeline"
I1112 08:44:52.620700    2788 apply_list.go:67] Installing "sample-templates/mongodb"
I1112 08:44:52.620987    2788 apply_list.go:67] Installing "sample-templates/mysql"
I1112 08:44:52.621255    2788 apply_list.go:67] Installing "sample-templates/postgresql"
I1112 08:44:52.621521    2788 apply_list.go:67] Installing "sample-templates/cakephp quickstart"
I1112 08:45:19.638190    2788 interface.go:41] Finished installing "sample-templates/mariadb" "sample-templates/dancer quickstart" "sample-templates/django quickstart" "sample-templates/rails quickstart" "sample-templates/jenkins pipeline ephemeral" "sample-templates/sample pipeline" "sample-templates/mongodb" "sample-templates/mysql" "sample-templates/postgresql" "sample-templates/cakephp quickstart" "sample-templates/nodejs quickstart"
I1112 08:45:46.546186    2788 interface.go:41] Finished installing "persistent-volumes" "openshift-web-console-operator" "openshift-router" "openshift-image-registry" "sample-templates" "centos-imagestreams"
Login to server ...
Creating initial project "myproject" ...
Server Information ...
OpenShift server started.

The server is accessible via web console at:
    https://192.168.64.74:8443/console

You are logged in as:
    User:     developer
    Password: <any value>

To login as administrator:
    oc login -u system:admin


OpenShift Started successfully

@codecov-io
Copy link

Codecov Report

Merging #2903 into master will decrease coverage by 2.26%.
The diff coverage is 5.26%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2903      +/-   ##
==========================================
- Coverage   28.14%   25.87%   -2.27%     
==========================================
  Files         148      149       +1     
  Lines        7724     7775      +51     
==========================================
- Hits         2174     2012     -162     
- Misses       5267     5509     +242     
+ Partials      283      254      -29
Impacted Files Coverage Δ
cmd/minishift/cmd/config/config.go 84.21% <ø> (ø) ⬆️
cmd/minishift/cmd/util/shared_flags.go 17.02% <0%> (-7.98%) ⬇️
cmd/minishift/cmd/addon/addon_remove.go 7.89% <0%> (ø) ⬆️
cmd/minishift/cmd/update.go 18.84% <0%> (ø) ⬆️
cmd/minishift/cmd/addon/addon_apply.go 19.44% <0%> (ø) ⬆️
cmd/minishift/cmd/util/util.go 3.09% <0%> (-0.62%) ⬇️
cmd/minishift/cmd/root.go 43.57% <100%> (ø) ⬆️
cmd/minishift/cmd/openshift/start.go 6.12% <6.12%> (ø)
cmd/minishift/cmd/start.go 15.26% <9.09%> (-4.91%) ⬇️
pkg/util/github/github.go 4.83% <0%> (-58.46%) ⬇️
... and 2 more

@amitkrout
Copy link
Contributor

Works fine and deployed the default nodejs app on the same running cluster. No failure as such

Tested on - fedora 29 + kvm

@amitkrout
Copy link
Contributor

@praveenkumar How this is differ from our regular minishift start. Is it going to replace the regular minishift start in recent future?

Can you please add a feature file or create test coverage issue, will take it in the current sprint.

@praveenkumar
Copy link
Contributor Author

How this is differ from our regular minishift start. Is it going to replace the regular minishift start in recent future?

@amitkrout This is not going to replace current minishift start in future also. This is just for our debug purpose and provides the user an extra way to edit their config file if required.

Can you please add a feature file or create test coverage issue, will take it in the current sprint.

@amitkrout I can create a follow-up the issue once it merges about the test coverage. Since it is just an experimental feature so not going to affect our generic workflow.

@agajdosi
Copy link

@praveenkumar I was expecting that --write-config will take some parameters to be added, written to config file. One can run it just as you mention in example code above: minishift start --write-config. So I am not sure, what that does, from where the written config is taken? How can I specify initialDelaySeconds: 500?

Thank you for help, feeling lost in it right now 😰 .

@amitkrout
Copy link
Contributor

amitkrout commented Nov 14, 2018

@agajdosi AFAIU minishift start --write-config writes the master config and node config to /var/lib/minishift/base including pulling the container images and keep the setup ready for oc cluster up. I don't think minishift openshift config set --patch would be helpful for patching origin config because it needs OpenShift instance to restart for patching to take effect, and also for other config file changes manual update is the only possibility i can see as of now. For example

  • minishift start --write-config
  • minishift ssh
  • sudo vi /var/lib/minishift/base/static-pod-manifests/apiserver.yaml
  • Updated the liveness probe as follows
livenessProbe:
        initialDelaySeconds: 500
        httpGet:
          scheme: HTTPS

And to consume this change run minishift openshift start.@praveenkumar correct me if i am wrong.

@praveenkumar
Copy link
Contributor Author

correct me if i am wrong.

@amitkrout that's right, @agajdosi so this is nothing to do with our openshift patch command stuff, it is mostly for debug purpose for the specific config change.

@LalatenduMohanty
Copy link
Member

Tested again and it is working fine.

@LalatenduMohanty LalatenduMohanty merged commit cce1a20 into minishift:master Dec 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants