Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

-bash: inspectXeption: line 0: syntax error near unexpected token `;' #31

Closed
tanpuekai opened this issue Nov 1, 2019 · 7 comments
Closed

-bash: inspectXeption: line 0: syntax error near unexpected token `;' #31

tanpuekai opened this issue Nov 1, 2019 · 7 comments

Comments

@tanpuekai
Copy link

Hi Jin,

Apart from my earlier question (#30) how to deal the issue that backend version ran for 24hrs before it crashed because of excessive job submission, I also found that even before that there are errors in the call-read-genome-tsv step:

$ find .|grep stderr|xargs ls -lS|head -n20
-rw------- 1 user group 1215 Nov  1 23:42 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stderr
-rw-r--r-- 1 user group    0 Nov  2 01:23 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-0/execution/stderr.check
-rw-r--r-- 1 user group    0 Nov  1 23:37 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-0/execution/stderr.submit
-rw-r--r-- 1 user group    0 Nov  2 01:24 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-1/execution/stderr.check
-rw-r--r-- 1 user group    0 Nov  1 23:37 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-1/execution/stderr.submit
-rw-r--r-- 1 user group    0 Nov  2 01:25 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-2/execution/stderr.check
-rw-r--r-- 1 user group    0 Nov  1 23:37 ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-align_mito/shard-2/execution/stderr.submit

The first stderr file above has non-zero size.
Below is the contents of the first stderr file above:

$ cat ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stderr
-bash: inspectXeption: line 0: syntax error near unexpected token `;'
-bash: inspectXeption: line 0: `inspectXeption () {  echo "***checking exceptions***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep ception|wc -l`; if [ $cnt -gt 0 ]; then;  echo $i '       ' $cnt; fi; done; echo ""; echo "***checking unfinished jobs***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep "Total time"|wc -l`; if [ $cnt -eq 0 ]; then;  echo $i '    ' $cnt; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_inspectXeption'
-bash: waitqueue: line 0: syntax error near unexpected token `;'
-bash: waitqueue: line 0: `waitqueue () {  while true; do;  sleep 10; cnt1=`qstat -rn1|grep user|wc -l`; if [ $cnt1 -gt 0 ]; then;  echo $cnt1; else;  break; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_waitqueue'
-bash: sedline: line 0: syntax error near unexpected token `;'
-bash: sedline: line 0: `sedline () {  for i in $(seq 1 10); do;  sed -i 's/  / /g' $1; done; sed -i 's/^ //g' $1; sed -i ':a;N;$!ba;s/\n /\n/g' $1; sed -i ':a;N;$!ba;s/ \n/\n/g' $1; sed -i 's/ /\t/g' $1; sed -i 's/\tCHR/CHR/g' $1; sed -i 's/^\t//g' $1; }'
-bash: error importing function definition for `BASH_FUNC_sedline'

This stderr was not found in my successfully finished run of your pipeline for individual bio-reps on local/non-backend CPUs.

What might be wrong? Is it a bug too?
Thanks.

Best
Chan
@leepc12
Copy link
Contributor

leepc12 commented Nov 1, 2019 •
edited

Can you make a TAR packing contents of this directory ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/?

Please upload the following items. This will be very helpful for debugging.

  1. TAR ball of ./atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/
  2. Full caper run command line.
  3. Your ~/.caper/default.conf.
  4. Your input JSON file (-i).
  5. Full caper run screen output (STDOUT).

@tanpuekai
Copy link
Author

1, See attached tar-ball
for.jin.lee.tar.gz

2, The full caper run command is as this:

 caper run \
                         -i input.json \
                         --tmp-dir /project1/TMPDIR4 \
                         --pbs-queue  legacy \
                         ../atac-seq-pipeline/atac.wdl

3, The ~/.caper/default.conf is:

$ cat ~/.caper/default.conf
backend=pbs

4, See attached tar-ball: for.jin.lee.tar.gz

5, See attached tar-ball: Message_encod4_335233.omics.txt for.jin.lee.tar.gz

Thanks.

Best
Chan

@leepc12
Copy link
Contributor

leepc12 commented Nov 3, 2019

Pipeline failed at the first task read_genome_tsv.
The following is STDERR from the failed task but I couldn't find anything related to the pipeline code or Caper.

-bash: inspectXeption: line 0: syntax error near unexpected token `;'
-bash: inspectXeption: line 0: `inspectXeption () {  echo "***checking exceptions***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep ception|wc -l`; if [ $cnt -gt 0 ]; then;  echo $i '	' $cnt; fi; done; echo ""; echo "***checking unfinished jobs***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep "Total time"|wc -l`; if [ $cnt -eq 0 ]; then;  echo $i '	' $cnt; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_inspectXeption'
-bash: waitqueue: line 0: syntax error near unexpected token `;'
-bash: waitqueue: line 0: `waitqueue () {  while true; do;  sleep 10; cnt1=`qstat -rn1|grep USER1|wc -l`; if [ $cnt1 -gt 0 ]; then;  echo $cnt1; else;  break; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_waitqueue'
-bash: sedline: line 0: syntax error near unexpected token `;'
-bash: sedline: line 0: `sedline () {  for i in $(seq 1 10); do;  sed -i 's/  / /g' $1; done; sed -i 's/^ //g' $1; sed -i ':a;N;$!ba;s/\n /\n/g' $1; sed -i ':a;N;$!ba;s/ \n/\n/g' $1; sed -i 's/ /\t/g' $1; sed -i 's/\tCHR/CHR/g' $1; sed -i 's/^\t//g' $1; }'
-bash: error importing function definition for `BASH_FUNC_sedline'

Can you run the following command and see what happens? This is a raw command line used by the pipeline to qsub this bash script /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/script for the failed task.

echo "/bin/bash /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/script" | qsub         -N cromwell_21385ee3_read_genome_tsv         -o /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stdout         -e /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stderr         -lselect=1:ncpus=1:mem=4000mb         -lwalltime=1:0:0                  -q legacy                  -V

@tanpuekai
Copy link
Author

Hi Jin,

Ok. This what I did:


$cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/
$echo "/bin/bash script" | qsub -N cromwell_21385ee3_read_genome_tsv -o STDOUT -e STDERR -lselect=1:ncpus=1:mem=4000mb -lwalltime=1:0:0 -q small -V
$ ls -lt|head
total 96
-rw------- 1 USER1 group 1276 Nov  3 13:01 STDERR
-rw------- 1 USER1 group    0 Nov  3 13:01 STDOUT
-rw-r--r-- 1 USER1 group    2 Nov  3 12:58 rc.tmp
-rw-r--r-- 1 USER1 group  150 Nov  3 12:58 roadmap_meta
$ cat STDERR
-bash: inspectXeption: line 0: syntax error near unexpected token `;'
-bash: inspectXeption: line 0: `inspectXeption () {  echo "***checking exceptions***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep ception|wc -l`; if [ $cnt -gt 0 ]; then;  echo $i '       ' $cnt; fi; done; echo ""; echo "***checking unfinished jobs***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep "Total time"|wc -l`; if [ $cnt -eq 0 ]; then;  echo $i '    ' $cnt; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_inspectXeption'
-bash: waitqueue: line 0: syntax error near unexpected token `;'
-bash: waitqueue: line 0: `waitqueue () {  while true; do;  sleep 10; cnt1=`qstat -rn1|grep USER1|wc -l`; if [ $cnt1 -gt 0 ]; then;  echo $cnt1; else;  break; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_waitqueue'
-bash: sedline: line 0: syntax error near unexpected token `;'
-bash: sedline: line 0: `sedline () {  for i in $(seq 1 10); do;  sed -i 's/  / /g' $1; done; sed -i 's/^ //g' $1; sed -i ':a;N;$!ba;s/\n /\n/g' $1; sed -i ':a;N;$!ba;s/ \n/\n/g' $1; sed -i 's/ /\t/g' $1; sed -i 's/\tCHR/CHR/g' $1; sed -i 's/^\t//g' $1; }'
-bash: error importing function definition for `BASH_FUNC_sedline'
/usr/bin/script: /usr/bin/script: cannot execute binary file
$

BTW, the contents of the file script is as follows:

$cat script
#!/bin/bash

cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
tmpDir=$(mkdir -p "/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/tmp.4b17423c" && echo "/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/tmp.4b17423c")

export _JAVA_OPTIONS=-Djava.io.tmpdir="$tmpDir"
export TMPDIR="$tmpDir"
export HOME="$HOME"
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution

)
out53879f59="${tmpDir}/out.$$" err53879f59="${tmpDir}/err.$$"
mkfifo "$out53879f59" "$err53879f59"
trap 'rm "$out53879f59" "$err53879f59"' EXIT
tee '/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stdout' < "$out53879f59" &
tee '/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stderr' < "$err53879f59" >&2 &
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution


# create empty files for all entries
touch genome_name
touch ref_fa bowtie2_idx_tar bwa_idx_tar chrsz gensz blacklist blacklist2
touch custom_aligner_idx_tar
touch ref_mito_fa
touch bowtie2_mito_idx_tar bwa_mito_idx_tar custom_aligner_mito_idx_tar
touch tss tss_enrich # for backward compatibility
touch dnase prom enh reg2map reg2map_bed roadmap_meta
touch mito_chr_name
touch regex_bfilt_peak_chr_name

python <<CODE
import os
with open('/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/inputs/1245739974/mm10_caper.local.tsv','r') as fp:
        for line in fp:
                arr = line.strip('\n').split('\t')
                if arr:
                        key, val = arr
                        with open(key,'w') as fp2:
                                fp2.write(val)
CODE
)  > "$out53879f59" 2> "$err53879f59"
echo $? > /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc.tmp
(
# add a .file in every empty directory to facilitate directory delocalization on the cloud
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
find . -type d -exec sh -c '[ -z "$(ls -A '"'"'{}'"'"')" ] && touch '"'"'{}'"'"'/.file' \;
)
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
sleep 30 && sync


)
mv /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc.tmp /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc
$

Thanks.

Best
Chan

@tanpuekai
Copy link
Author

Hi Jin,

Seems the problem is in the directive -V in your command:

"/bin/bash script" | qsub -N cromwell_21385ee3_read_genome_tsv -o STDOUT -e STDERR -lselect=1:ncpus=1:mem=4000mb -lwalltime=1:0:0 -q small -V

I changed the file script to:

#!/bin/bash
#PBS -N cromwell_21385ee3_read_genome_tsv
#PBS -o STDOUT
#PBS -e STDERR
#PBS -lselect=1:ncpus=12:mem=40000mb
#PBS -lwalltime=24:0:0
#PBS -q medium
#PBS -V
#PBS -j oe

date
echo "Step 1"
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
tmpDir=$(mkdir -p "/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/tmp.4b17423c" && echo "/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/tmp.4b17423c")

export _JAVA_OPTIONS=-Djava.io.tmpdir="$tmpDir"
export TMPDIR="$tmpDir"
export HOME="$HOME"
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution

)
out53879f59="${tmpDir}/out.$$" err53879f59="${tmpDir}/err.$$"
mkfifo "$out53879f59" "$err53879f59"
trap 'rm "$out53879f59" "$err53879f59"' EXIT
tee '/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stdout' < "$out53879f59" &
tee '/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/stderr' < "$err53879f59" >&2 &
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution

echo "Step 2"
# create empty files for all entries
touch genome_name
touch ref_fa bowtie2_idx_tar bwa_idx_tar chrsz gensz blacklist blacklist2
touch custom_aligner_idx_tar
touch ref_mito_fa
touch bowtie2_mito_idx_tar bwa_mito_idx_tar custom_aligner_mito_idx_tar
touch tss tss_enrich # for backward compatibility
touch dnase prom enh reg2map reg2map_bed roadmap_meta
touch mito_chr_name
touch regex_bfilt_peak_chr_name

echo "Step 3"
python <<CODE
import os
with open('/project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/inputs/1245739974/mm10_caper.local.tsv','r') as fp:
        for line in fp:
                arr = line.strip('\n').split('\t')
                if arr:
                        key, val = arr
                        with open(key,'w') as fp2:
                                fp2.write(val)
CODE
)  > "$out53879f59" 2> "$err53879f59"
echo "Step 4"
echo $? > /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc.tmp
(
# add a .file in every empty directory to facilitate directory delocalization on the cloud
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
find . -type d -exec sh -c '[ -z "$(ls -A '"'"'{}'"'"')" ] && touch '"'"'{}'"'"'/.file' \;
)
(
cd /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution
sleep 30 && sync


)
mv /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc.tmp /project1/second-bat-one-go/atac/21385ee3-a703-4a97-8a87-855e8941fa03/call-read_genome_tsv/execution/rc
date
echo finished

and then performed this:

$ qsub script

and in 1-2 minutes it is finished:

$ ls -lt [Ss]*|head
-rw------- 1 USER1 group 3714 Nov  3 13:51 STDOUT
-rw-r--r-- 1 USER1 group 2418 Nov  3 13:51 stdout
-rw-r--r-- 1 USER1 group    0 Nov  3 13:51 stderr
-rw-r--r-- 1 USER1 group 3642 Nov  3 13:50 script
$ cat STDOUT
-bash: inspectXeption: line 0: syntax error near unexpected token `;'
-bash: inspectXeption: line 0: `inspectXeption () {  echo "***checking exceptions***"; for i in `ls *reeting*`; do;  cnt=`cat $i|grep ception|wc -l`; if [ $cnt -gt 0 ]; then;  echo $i '       ' $cnt; fi; done; echo ""; echo "***checking
-bash: error importing function definition for `BASH_FUNC_inspectXeption'
-bash: waitqueue: line 0: syntax error near unexpected token `;'
-bash: waitqueue: line 0: `waitqueue () {  while true; do;  sleep 10; cnt1=`qstat -rn1|grep USER1|wc -l`; if [ $cnt1 -gt 0 ]; then;  echo $cnt1; else;  break; fi; done; }'
-bash: error importing function definition for `BASH_FUNC_waitqueue'
-bash: sedline: line 0: syntax error near unexpected token `;'
-bash: sedline: line 0: `sedline () {  for i in $(seq 1 10); do;  sed -i 's/  / /g' $1; done; sed -i 's/^ //g' $1; sed -i ':a;N;$!ba;s/\n /\n/g' $1; sed -i ':a;N;$!ba;s/ \n/\n/g' $1; sed -i 's/ /\t/g' $1; sed -i 's/\tCHR/CHR/g' $1; sed -
-bash: error importing function definition for `BASH_FUNC_sedline'
Sun Nov  3 13:51:02 UTC 2019
Step 1
Step 2
Step 2
Step 3
Sun Nov  3 13:51:32 UTC 2019
finished

the file stdout has the same contents as STDOUT, except the -bash: inspectXeption: part.

Thanks.

Best
Chan

@tanpuekai
Copy link
Author

Hi Jin,

I guess I know where the problem is.

waitqueue (), sedline (), and inspectXeption () are three self-defined functions in my ~/.bash_profile, wherein I also have these lines:

export -f sedline
export -f waitqueue inspectXeption

and your -V is basically asking our PBS server to take my self-defined functions as well. Somehow, our PBS server has a different environment as its login node, i.e. where my home folder is. So that I can log in and use these self-defined functions, but in the PBS server mode, they can't be defined.

I commented these 3 functions, and it is all ok now.

Thanks.

Best
Chan

@tanpuekai
Copy link
Author

problem solved:

$ find .|grep stderr|xargs ls -lt
-rw-r--r-- 1 USER1 group 0 Nov  3 15:09 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-4/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:09 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-1/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:09 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-3/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:06 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-2/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:06 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-5/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:06 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-0/execution/stderr.check
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-2/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-2/execution/stderr.submit
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-0/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-0/execution/stderr.submit
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-4/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-4/execution/stderr.submit
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-5/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-5/execution/stderr.submit
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-3/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-3/execution/stderr.submit
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-1/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:02 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-align/shard-1/execution/stderr.submit
-rw------- 1 USER1 group 0 Nov  3 15:01 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-read_genome_tsv/execution/stderr
-rw-r--r-- 1 USER1 group 0 Nov  3 15:01 ./atac/efd3d433-78df-41c8-93ad-d180c753a733/call-read_genome_tsv/execution/stderr.submit

Thanks.

Best
Chan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tanpuekai @leepc12