Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

サーバーに設定する #21

Closed
murachi opened this issue Mar 28, 2020 · 24 comments · Fixed by #32
Closed

サーバーに設定する #21

murachi opened this issue Mar 28, 2020 · 24 comments · Fixed by #32
Assignees
Milestone

Comments

@murachi
Copy link
Owner

murachi commented Mar 28, 2020

はらぺこ鯖に設定してみる。上手く動かない部分があればバグ修正する。

@murachi murachi added this to the first release milestone Mar 28, 2020
@murachi murachi self-assigned this Mar 28, 2020
@murachi
Copy link
Owner Author

murachi commented May 3, 2020

uWSGI を使おうかと思ってたんだが、 Trac 用に既に入れててそっちは Python2 。 conflict してややこしいことになる悪寒(´・_・`) そこは気をつけないと…

@murachi
Copy link
Owner Author

murachi commented May 3, 2020

pipenv とか使ったほうがいいのかなぁ…(´・_・`)

murachi added a commit that referenced this issue May 5, 2020
@murachi
Copy link
Owner Author

murachi commented May 5, 2020

poetry の存在とかすっかり忘れてたけどアプリケーションの環境固定用であれば pipenv が一番お手頃かつ十分な気がしたので pipenv で環境作ってみた。

実際動かすには su してから pipenv install または pipenv sync し、 root ユーザー上で環境を構成した状態で pipenv shell してから uwsgi を UID 付きで実行する。

murachi@maha:~/github/tmpbox$ sudo pipenv sync
murachi@maha:~/github/tmpbox$ cd src
murachi@maha:~/github/tmpbox/src$ sudo pipenv shell
(tmpbox) root@maha:~/github/tmpbox/src# uwsgi --http=0.0.0.0:6543 --wsgi-file=tmpbox.py --callable=app --uid=tmpbox

@murachi
Copy link
Owner Author

murachi commented May 9, 2020

uWSGI 用の設定ファイルも setup.py にて生成するようにしてみた。

pipenv からだと UID を指定してのデバッグ実行が flask コマンドからだと上手く出来なさそうだったので、 UID の指定も出来る uWSGI を使用する想定で、デバッグ実行用の設定ファイルも出力するようにしている。これまで、デバッグ実行時は以下のようにコマンド実行していたが、

murachi@maha:~/github/tmpbox$ cd src
murachi@maha:~/github/tmpbox/src$ sudo su tmpbox -c ./run.sh

pipenv を用いる場合は以下の方法でデバッグ実行できるようになる。

murachi@maha:~/github/tmpbox$ sudo pipenv run uwsgi --ini src/conf.d/uwsgi-debug.ini

@murachi
Copy link
Owner Author

murachi commented May 9, 2020

一旦 merge しまっす。

murachi added a commit that referenced this issue May 9, 2020
引き続き #21 にてサーバーへの設定作業を行います…。
@murachi
Copy link
Owner Author

murachi commented May 9, 2020

えー。鯖設定を行っているのですが、 setup.py 実行中に MySQL への SQL 発行でコケました orz

管理者アカウントの設定を行います。
ユーザーID >> murachi
表示名 >> T.MURACHI
パスワード >> 
念のため、もう一度 >> 
Traceback (most recent call last):
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1247, in _execute_context
    self.dialect.do_execute(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 517, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 732, in _read_query_result
    result.read()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 1075, in read
    first_packet = self.connection._read_packet()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 684, in _read_packet
    packet.check_error()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CURRENT_DATE, \n\tsummary TEXT, \n\texpires_days INTEGER NOT NULL, \n\tis_deleted BOOL' at line 4")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 304, in <module>
    db.create_tables()
  File "src/tmpbox_db_accessor.py", line 415, in create_tables
    Base.metadata.create_all(self.engine)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4320, in create_all
    bind._run_visitor(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2058, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1627, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 144, in traverse_single
    return meth(obj, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 777, in visit_metadata
    self.traverse_single(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 144, in traverse_single
    return meth(obj, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 821, in visit_table
    self.connection.execute(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 984, in execute
    return meth(self, multiparams, params)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1041, in _execute_ddl
    ret = self._execute_context(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1287, in _execute_context
    self._handle_dbapi_exception(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _handle_dbapi_exception
    util.raise_(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1247, in _execute_context
    self.dialect.do_execute(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 517, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 732, in _read_query_result
    result.read()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 1075, in read
    first_packet = self.connection._read_packet()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/connections.py", line 684, in _read_packet
    packet.check_error()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CURRENT_DATE, \n\tsummary TEXT, \n\texpires_days INTEGER NOT NULL, \n\tis_deleted BOOL' at line 4")
[SQL: 
CREATE TABLE directory (
	directory_id INTEGER NOT NULL AUTO_INCREMENT, 
	directory_name VARCHAR(100) NOT NULL, 
	create_date DATE NOT NULL DEFAULT CURRENT_DATE, 
	summary TEXT, 
	expires_days INTEGER NOT NULL, 
	is_deleted BOOL NOT NULL DEFAULT false, 
	PRIMARY KEY (directory_id), 
	CHECK (is_deleted IN (0, 1))
)

]
(Background on this error at: http://sqlalche.me/e/f405)

@murachi
Copy link
Owner Author

murachi commented May 9, 2020

なるほどなるほど。

directory.expires_days はデフォルト値に functions.current_date() を使っていて、これは MySQL (PyMySQL) の場合、SQL 文中に CURRENT_DATE を挿入する。しかしながら、 MySQL ではカッコ付きではない CURRENT_DATE は例えば Select 文中などでは使用できるが、 CREATE TABLE 中のフィールドのデフォルト値としてはカッコ付きの CURRENT_DATE() は使用できるものの、カッコなしの CURRENT_DATE は使用できないらしい。そのために結果として SQL が Syntax error になってしまっている。

どこかしらのライブラリベンダーにいちゃもんつけて修正してもらいたいレベルの案件ですが、しゃーないので回避策を探るしかなさそう… orz

@murachi
Copy link
Owner Author

murachi commented May 10, 2020

嘘やったorz CURRENT_DATE() でもダメじゃん…

MariaDB [minimum]> create table hoge (
    ->   hoge_id int not null primary key
    -> , create_date date not null default current_date
    -> );
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'current_date
)' at line 3
MariaDB [minimum]> create table hoge (
    ->   hoge_id int not null primary key
    -> , create_date date not null default current_date()
    -> );
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'current_date()
)' at line 3
MariaDB [minimum]> 

@murachi
Copy link
Owner Author

murachi commented May 11, 2020

もともと出ていたエラーの原因は MariaDB のバージョンが古いことでした (CREATE TABLEDEFAULT に関数等のフルサポートが導入されたのが10.2.1 以降とのこと)。

で、MariaDB のバージョンを上げてリトライしたところ、今度はまた別のエラーが…(´・_・`)

管理者アカウントの設定を行います。
ユーザーID >> murachi
表示名 >> T.MURACHI
パスワード >> 
念のため、もう一度 >> 
Traceback (most recent call last):
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 2894, in visit_create_table
    processed = self.process(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 350, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 95, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 2928, in visit_create_column
    text = self.get_column_specification(column, first_pk=first_pk)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 1573, in get_column_specification
    self.dialect.type_compiler.process(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 400, in process
    return type_._compiler_dispatch(self, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 95, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 3419, in visit_unicode
    return self.visit_VARCHAR(type_, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py", line 2046, in visit_VARCHAR
    raise exc.CompileError(
sqlalchemy.exc.CompileError: VARCHAR requires a length on dialect mysql

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 304, in <module>
    db.create_tables()
  File "src/tmpbox_db_accessor.py", line 415, in create_tables
    Base.metadata.create_all(self.engine)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4320, in create_all
    bind._run_visitor(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2058, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1627, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 144, in traverse_single
    return meth(obj, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 777, in visit_metadata
    self.traverse_single(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 144, in traverse_single
    return meth(obj, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 821, in visit_table
    self.connection.execute(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 984, in execute
    return meth(self, multiparams, params)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1035, in _execute_ddl
    compiled = ddl.compile(
  File "<string>", line 1, in <lambda>
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 468, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 29, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 319, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 350, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 95, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/sql/compiler.py", line 2904, in visit_create_table
    util.raise_(
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
sqlalchemy.exc.CompileError: (in table 'file_info', column 'summary'): VARCHAR requires a length on dialect mysql

@murachi
Copy link
Owner Author

murachi commented May 11, 2020

色々とバグ修正。で、デバッグモードで動かせるようになったが、ログインした時点で以下のエラーが発生…。

[ERROR] debug: Not handled error was raised.
Traceback (most recent call last):
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "tmpbox.py", line 152, in page_index
    login_session, _ = verify_login_session()
  File "tmpbox.py", line 81, in verify_login_session
    login_session = db.check_login_session(session.get('token', None))
  File "./tmpbox_db_accessor.py", line 584, in check_login_session
    return self.session_scope(
  File "./tmpbox_db_accessor.py", line 393, in session_scope
    result = query_func(session)
  File "./tmpbox_db_accessor.py", line 585, in <lambda>
    lambda s: self.__session_check_login_session(s, session_id), True)
  File "./tmpbox_db_accessor.py", line 596, in __session_check_login_session
    login_session = SessionState.filter_check_expires(
  File "./tmpbox_db_accessor.py", line 145, in filter_check_expires
    func.timestampdiff(text('minute'), SessionState.access_dt, functions.now())
NameError: name 'func' is not defined
Traceback (most recent call last):
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 1953, in full_dispatch_request
    return self.finalize_request(rv)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 1968, in finalize_request
    response = self.make_response(rv)
  File "/home/murachi/.local/share/virtualenvs/tmpbox-uUfjWWGP/lib/python3.8/site-packages/flask/app.py", line 2127, in make_response
    raise TypeError(
TypeError: The view function did not return a valid response. The return type must be a string, dict, tuple, Response instance, or WSGI callable, but it was a NameError.
[pid: 25451|app: 0|req: 10/10] 61.125.220.180 () {40 vars in 736 bytes} [Mon May 11 19:14:04 2020] GET / => generated 0 bytes in 8 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)

@murachi
Copy link
Owner Author

murachi commented May 13, 2020

uWSGI のデバッグモードでファイルのアップロードまで動くのを確認。

@murachi
Copy link
Owner Author

murachi commented May 13, 2020

残作業:

  • uWSGI のデーモンモードが動作するよう systemd を設定。
  • Nginx との連携設定。

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

  • uWSGI のデーモンモードでの動作は確認済み。
  • Nginx と uWSGI の連携動作も確認済み。
  • uWSGI を systemd に登録し、 systemctl から起動すると、何故か socket ファイルが生成されず、 Nginx との連携が取れなくなる。

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

訳がわからない。いや、分かりかけてきてはいるんだけど、再現性が安定しない。

root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 8
drwxr-s--- 2 www-data www-data 4096  5月 14 11:18 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
root@ik1-314-17353:~# systemctl start tmpbox.service
root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 12
drwxr-s--- 2 www-data www-data 4096  5月 14 11:18 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
-rw-rw-rw- 1 root     www-data    6  5月 14 11:18 uwsgi-tmpbox.pid
srw-rw-rw- 1 www-data www-data    0  5月 14 11:18 uwsgi-tmpbox.sock
root@ik1-314-17353:~# systemctl stop tmpbox.service
root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 12
drwxr-s--- 2 www-data www-data 4096  5月 14 11:18 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
-rw-rw-rw- 1 root     www-data    6  5月 14 11:18 uwsgi-tmpbox.pid
root@ik1-314-17353:~# systemctl start tmpbox.service
root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 12
drwxr-s--- 2 www-data www-data 4096  5月 14 11:19 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
-rw-rw-rw- 1 root     www-data    6  5月 14 11:19 uwsgi-tmpbox.pid
srw-rw-rw- 1 www-data www-data    0  5月 14 11:19 uwsgi-tmpbox.sock
root@ik1-314-17353:~# systemctl stop tmpbox.service
root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 12
drwxr-s--- 2 www-data www-data 4096  5月 14 11:19 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
-rw-rw-rw- 1 root     www-data    6  5月 14 11:19 uwsgi-tmpbox.pid
root@ik1-314-17353:~# systemctl start tmpbox.service
root@ik1-314-17353:~# ls -la /var/tmpbox/run/
total 12
drwxr-s--- 2 www-data www-data 4096  5月 14 11:20 .
drwxr-s--- 6 www-data www-data 4096  5月 13 16:46 ..
-rw-rw-rw- 1 root     www-data    6  5月 14 11:20 uwsgi-tmpbox.pid
root@ik1-314-17353:~# 
  • PID ファイルは root 権限で生成される。 (どうやら Trac でもそうだったらしい)
  • サービス停止時に PID ファイルが削除されない。
  • 再開時に uWSGI 側で vacuum の設定が効いて PID が削除されるはずだが、権限の問題で削除されないのかもしれない?
    • そうだとすると Trac の起動も不安定になりそうなものだけど、そういうわけでもなく…

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

root@ik1-314-17353:~# systemctl reload tmpbox.service
root@ik1-314-17353:~# tail -n 20 /var/tmpbox/log/uwsgi-tmpbox.log 
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/tmpbox/run/uwsgi-tmpbox.sock fd 6
Python version: 2.7.12 (default, Apr 15 2020, 17:07:12)  [GCC 5.4.0 20160609]
Python main interpreter initialized at 0x149def0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
  File "/home/murachi/github/tmpbox/src/tmpbox.py", line 36
SyntaxError: Non-ASCII character '\xe8' in file /home/murachi/github/tmpbox/src/tmpbox.py on line 37, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
failed to parse file /home/murachi/github/tmpbox/src/tmpbox.py
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 24852)
spawned uWSGI worker 1 (pid: 24853, cores: 4)
root@ik1-314-17353:~# 

Python 2.7.12 …だと…

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

systemd の設定ですが、

[Unit]
Description=Tmpbox service on uWSGI
After=network.target
ConditionPathExists=/home/murachi/github/tmpbox

[Service]
WorkingDirectory=/home/murachi/github/tmpbox
ExecStart=/usr/local/bin/pipenv run uwsgi --ini src/conf.d/uwsgi-daemon.ini
ExecReload=/usr/local/bin/pipenv run uwsgi --stop /var/tmpbox/run/uwsgi-tmpbox.pid
ExecStop=/usr/local/bin/pipenv run uwsgi --stop /var/tmpbox/run/uwsgi-tmpbox.pid
Restart=on-failure
Type=forking

[Install]
WantedBy=multi-user.target

WorkingDirectory を設定していて、相対パス指定で uwsgi-daemon.ini ファイルも見えている、にもかかわらず pipenv が対応するバージョンの python 環境にアクセスできていない?

systemd から pipenv を使う場合のコツとかがあるのかな…(´・_・`)

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

やっと原因わかった。

murachi@ik1-314-17353:~/github/tmpbox$ sudo pipenv run uwsgi --ini src/conf.d/uwsgi-daemon.ini 
[sudo] murachi のパスワード: 
[uWSGI] getting INI configuration from src/conf.d/uwsgi-daemon.ini
murachi@ik1-314-17353:~/github/tmpbox$ sudo tail -n 20 /var/tmpbox/log/uwsgi-tmpbox.log
setgid() to 33
setuid() to 33
chdir() to /home/murachi/github/tmpbox/src
your processes number limit is 7843
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/tmpbox/run/uwsgi-tmpbox.sock fd 6
Python version: 3.8.2 (default, Feb 26 2020, 04:23:39)  [GCC 5.4.0 20160609]
Python main interpreter initialized at 0x26f66f0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x26f66f0 pid: 25027 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 25027)
spawned uWSGI worker 1 (pid: 25031, cores: 4)
murachi@ik1-314-17353:~/github/tmpbox$ sudo pipenv run uwsgi --stop /var/tmpbox/run/uwsgi-tmpbox.pid
murachi@ik1-314-17353:~/github/tmpbox$ sudo su -
root@ik1-314-17353:~# cd /home/murachi/github/tmpbox/
root@ik1-314-17353:/home/murachi/github/tmpbox# pipenv run uwsgi --ini src/conf.d/uwsgi-daemon.ini 
[uWSGI] getting INI configuration from src/conf.d/uwsgi-daemon.ini
root@ik1-314-17353:/home/murachi/github/tmpbox# tail -n 20 /var/tmpbox/log/uwsgi-tmpbox.log 
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/tmpbox/run/uwsgi-tmpbox.sock fd 6
Python version: 2.7.12 (default, Apr 15 2020, 17:07:12)  [GCC 5.4.0 20160609]
Python main interpreter initialized at 0x78d160
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
  File "/home/murachi/github/tmpbox/src/tmpbox.py", line 36
SyntaxError: Non-ASCII character '\xe8' in file /home/murachi/github/tmpbox/src/tmpbox.py on line 37, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
failed to parse file /home/murachi/github/tmpbox/src/tmpbox.py
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 25092)
spawned uWSGI worker 1 (pid: 25093, cores: 4)
root@ik1-314-17353:/home/murachi/github/tmpbox# pipenv run uwsgi --stop /var/tmpbox/run/uwsgi-tmpbox.pid 
root@ik1-314-17353:/home/murachi/github/tmpbox# 

インストールしたユーザー環境から sudo pipenv uwsgi で起動する分にはインストール時の pipenv 環境が使われるが、 sudo su - して root ユーザー環境に移行した状態で、ディレクトリだけ移動して pipenv uwsgi しても、ユーザー環境からインストールした pipenv 環境にはアクセスできない、ということらしい。何という罠…((((;゚Д゚))))ガクガクブルブル

そんなわけで、セットアップ時の手順として、

$ sudo pipenv sync

するのは間違いで、実は

$ sudo su -
# cd /home/account/path/to/project/
# pipenv sync

としないといけなかった、ということっぽい。(未確認)

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

root 環境から pipenv sync したところ、ちゃんと Python 3.8 が使われるようにはなったが、今度はなぜか flask が入っていないというエラーに… ('A`)

root@ik1-314-17353:/home/murachi/github/tmpbox# tail -n 30 /var/tmpbox/log/uwsgi-tmpbox.log 
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 33
setuid() to 33
chdir() to /home/murachi/github/tmpbox/src
your processes number limit is 7843
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/tmpbox/run/uwsgi-tmpbox.sock fd 6
Python version: 3.8.2 (default, Feb 26 2020, 04:23:39)  [GCC 5.4.0 20160609]
Python main interpreter initialized at 0x1c6bf20
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
Traceback (most recent call last):
  File "/home/murachi/github/tmpbox/src/tmpbox.py", line 3, in <module>
    from flask import Flask, url_for, render_template, redirect, abort, send_file, Markup, request, session
ModuleNotFoundError: No module named 'flask'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 25973)
spawned uWSGI worker 1 (pid: 25974, cores: 4)
--- no python application found, check your startup logs for errors ---
[pid: 25974|app: -1|req: -1/1] 61.125.220.180 () {44 vars in 694 bytes} [Thu May 14 12:09:01 2020] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
--- no python application found, check your startup logs for errors ---
[pid: 25974|app: -1|req: -1/2] 61.125.220.180 () {44 vars in 694 bytes} [Thu May 14 12:09:03 2020] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
root@ik1-314-17353:/home/murachi/github/tmpbox# 

なんでだ…

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

訳がわからない。 root 環境から pipenv run uwsgi すると flask モジュールが見つからないのに、 pipenv run python して対話モードで from flask import Flask する分には全く問題なく flask モジュールがロードされる。何が違うんだ??

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

とりあえずどういう現象なのかは判明。再現手順は以下の通り。

  1. sudo su - して root ユーザーになる。
  2. pipenv 環境のあるプロジェクトディレクトリへ移動し、 pipenv shell で pipenv 環境に入る。
  3. uwsgi コマンドを --uid パラメータ付きで実行する。
    • 起動するプロセス内で sys.path を参照すると、 pipenv 環境のものではないリストになっていることが確認できる。

インストールしたユーザーから sudo pipenv shell にて入った環境で --uid 付きで uwsgi を呼んだ場合のスモールケース:
image

root ユーザーから pipenv shell 似て入った環境で --uid 付きで uwsgi を呼んだ場合のスモールケース:
image

解決策は今のところちょっと思いつかない(´・_・`)

@murachi
Copy link
Owner Author

murachi commented May 14, 2020

multiprocessing モジュールを使って spawn 方式でプロセスを起動する際に同様の現象が確認できた。uwsgi のデフォルトももしかしたら spawn なのかも。これを fork に変える方法はないだろうか…

@murachi
Copy link
Owner Author

murachi commented May 16, 2020

環境変数 PIPENV_VENV_IN_PROJECT を真値にセットしておくと、 pipenv の仮想環境がプロジェクトディレクトリ内に生成されるようになる。これを使うと、現在の uid が何かに関わらず、常に同じ仮想環境が参照されるようになる (os.setuid() しても python コマンドのパスやライブラリパスに影響を与えない)。

とりあえずこの環境変数を試してみることにします…。

@murachi
Copy link
Owner Author

murachi commented May 17, 2020

できた!!
https://tmpbox.harapeko.jp/

murachi added a commit that referenced this issue May 17, 2020
issue #21 - pipenv の仮想環境をプロジェクトディレクトリ内に生成するよう修正。
@murachi
Copy link
Owner Author

murachi commented May 17, 2020

#22 の作業はこの issue を基にやりましょう…。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant