Deactivated packages listed on website #97

mreid-tt · 2023-04-26T11:47:38Z

In the website URL https://synocommunity.com/package/[package] there is listed some packages which are listed even though they are deactivated in the admin interface. Looking at SickBeard we see the following:

Admin status:

Website status:

We should not list versions that are not active. Moreover we should not list packages if none of their versions are active.

EDIT: Looking at the code, apparently the second issue was something known as missing:

spkrepo/spkrepo/views/frontend.py

Lines 98 to 99 in 76951e0

    
           def package(name): 
        
               # TODO: show only packages with at least a version and an active build

The text was updated successfully, but these errors were encountered:

mreid-tt · 2023-04-27T02:47:31Z

@Diaoul, I've authored two PRs (#98, #99) which together should resolve this issue. I'd appreciate your thoughts on my approach.

mreid-tt · 2023-04-27T19:50:15Z

hey @Diaoul, I'm not very familiar with the back-end system. What are the next steps to integrating these PRs into the production system?

Diaoul · 2023-04-29T07:50:06Z

I think we need to create a new release. I have GitHub actions to push the relevant docker images. Then on the server a few docker-compose commands should do the trick.

mreid-tt · 2023-05-03T12:18:14Z

I think we need to create a new release. I have GitHub actions to push the relevant docker images. Then on the server a few docker-compose commands should do the trick.

Hey @Diaoul, I'm not very familiar with releases but I've tried to create a draft (https://github.com/SynoCommunity/spkrepo/releases/tag/untagged-d15132cea06383933b7b). Let me know what you think. It asks for binaries to attach but with the build workflow not working do I just attach the zip file for the codebase?

Diaoul · 2023-05-03T16:32:19Z

It's building

mreid-tt · 2023-05-03T16:38:50Z

It's building

Exciting! It looks like there needs to be a version number change somewhere else... the logs seem to be showing an upload attempt of the old version number:

Uploading spkrepo-0.2.2-py3-none-any.whl 100%
HTTP Error 400: File already exists.

Diaoul · 2023-05-03T16:40:35Z

Here. My bad I did not check as the tag was present.

Diaoul · 2023-05-03T16:42:45Z

Done, it's building again

Diaoul · 2023-05-03T16:51:46Z

I'll run the update on the server later tonight.

Diaoul · 2023-05-03T19:18:02Z

Upgraded the server. Can you check it is working as you expect?

mreid-tt · 2023-05-03T19:32:45Z

Upgraded the server. Can you check it is working as you expect?

I may have to give some time for the caches to update to confirm. Right now I suspect the back-end is a bit busy because I'm getting a mix of "Error 503 first byte timeout" and "Internal Server Error".

EDIT: There were also a few other commits since the last release which I don't see PRs for so I hope these aren't causing issues now that the new release is out (which should include these):

987e918
645918e
4640a22
43e5266
76951e0

Diaoul · 2023-05-03T20:19:26Z


Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 136, in handle
    self.handle_request(listener, req, client, addr)
  File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 179, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.9/site-packages/werkzeug/middleware/proxy_fix.py", line 187, in __call__
    return self.app(environ, start_response)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2076, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1514, in full_dispatch_request
    rv = self.preprocess_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1857, in preprocess_request
    rv = self.ensure_sync(before_func)()
  File "/usr/local/lib/python3.9/site-packages/flask_principal.py", line 477, in _on_before_request
    identity = loader()
  File "/usr/local/lib/python3.9/site-packages/flask_security/core.py", line 245, in _identity_loader
    if not isinstance(current_user._get_current_object(), AnonymousUserMixin):
  File "/usr/local/lib/python3.9/site-packages/werkzeug/local.py", line 565, in _get_current_object
    return self.__local()  # type: ignore
  File "/usr/local/lib/python3.9/site-packages/flask_login/utils.py", line 26, in <lambda>
    current_user = LocalProxy(lambda: _get_user())
  File "/usr/local/lib/python3.9/site-packages/flask_login/utils.py", line 346, in _get_user
    current_app.login_manager._load_user()
  File "/usr/local/lib/python3.9/site-packages/flask_login/login_manager.py", line 318, in _load_user
    user = self._user_callback(user_id)
  File "/usr/local/lib/python3.9/site-packages/flask_security/core.py", line 221, in _user_loader
    return _security.datastore.find_user(id=user_id)
  File "/usr/local/lib/python3.9/site-packages/flask_security/datastore.py", line 254, in find_user
    return self.user_model.query.filter_by(**kwargs).first()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2810, in first
    return self.limit(1)._iter().first()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2894, in _iter
    result = self.session.execute(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1692, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1620, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1487, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1851, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2032, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1808, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column user.fs_uniquifier does not exist
LINE 2: ....github_access_token AS user_github_access_token, "user".fs_...```

mreid-tt · 2023-05-03T22:05:40Z

This looks like the commit for 76951e0. Did you roll it back?

EDIT: Or perhaps it could be this:

spkrepo/spkrepo/models.py

Line 38 in 43e5266

fs_uniquifier = db.Column(db.String(255), unique=True, nullable=False)

Which was implemented as part of 43e5266.

mreid-tt · 2023-05-03T22:13:51Z

EDIT: There were also a few other commits since the last release which I don't see PRs for so I hope these aren't causing issues now that the new release is out (which should include these):

987e918
645918e
4640a22
43e5266
76951e0

@publicarray, can you comment on these commits? Based on the error above, one of these may be causing an error on the server.

Diaoul · 2023-05-04T07:43:05Z

I've pushed a wip branch that I had locally that fixes an issue with the latest flask-security-too, maybe it's worth getting it some love. Currently some tests are failing but I think that's fixable.

publicarray · 2023-05-10T20:35:41Z




Traceback (most recent call last):

  File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 136, in handle

    self.handle_request(listener, req, client, addr)

  File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 179, in handle_request

    respiter = self.wsgi(environ, resp.start_response)

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2091, in __call__

    return self.wsgi_app(environ, start_response)

  File "/usr/local/lib/python3.9/site-packages/werkzeug/middleware/proxy_fix.py", line 187, in __call__

    return self.app(environ, start_response)

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2076, in wsgi_app

    response = self.handle_exception(e)

  File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router

    return original_handler(e)

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app

    response = self.full_dispatch_request()

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request

    rv = self.handle_user_exception(e)

  File "/usr/local/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router

    return original_handler(e)

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1514, in full_dispatch_request

    rv = self.preprocess_request()

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1857, in preprocess_request

    rv = self.ensure_sync(before_func)()

  File "/usr/local/lib/python3.9/site-packages/flask_principal.py", line 477, in _on_before_request

    identity = loader()

  File "/usr/local/lib/python3.9/site-packages/flask_security/core.py", line 245, in _identity_loader

    if not isinstance(current_user._get_current_object(), AnonymousUserMixin):

  File "/usr/local/lib/python3.9/site-packages/werkzeug/local.py", line 565, in _get_current_object

    return self.__local()  # type: ignore

  File "/usr/local/lib/python3.9/site-packages/flask_login/utils.py", line 26, in <lambda>

    current_user = LocalProxy(lambda: _get_user())

  File "/usr/local/lib/python3.9/site-packages/flask_login/utils.py", line 346, in _get_user

    current_app.login_manager._load_user()

  File "/usr/local/lib/python3.9/site-packages/flask_login/login_manager.py", line 318, in _load_user

    user = self._user_callback(user_id)

  File "/usr/local/lib/python3.9/site-packages/flask_security/core.py", line 221, in _user_loader

    return _security.datastore.find_user(id=user_id)

  File "/usr/local/lib/python3.9/site-packages/flask_security/datastore.py", line 254, in find_user

    return self.user_model.query.filter_by(**kwargs).first()

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2810, in first

    return self.limit(1)._iter().first()

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2894, in _iter

    result = self.session.execute(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1692, in execute

    result = conn._execute_20(statement, params or {}, execution_options)

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1620, in _execute_20

    return meth(self, args_10style, kwargs_10style, execution_options)

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection

    return connection._execute_clauseelement(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1487, in _execute_clauseelement

    ret = self._execute_context(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1851, in _execute_context

    self._handle_dbapi_exception(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2032, in _handle_dbapi_exception

    util.raise_(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_

    raise exception

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1808, in _execute_context

    self.dialect.do_execute(

  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute

    cursor.execute(statement, parameters)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column user.fs_uniquifier does not exist

LINE 2: ....github_access_token AS user_github_access_token, "user".fs_...```

Looks like the fs_uniquifirr column needs to be added to the database

Diaoul · 2023-05-16T07:48:36Z

Weird, I think I reverted the deployment and stuck to the old release instead 🤔

th0ma7 · 2023-05-16T09:47:19Z

That would explain why my python updates still showed up while not yet activated...

mreid-tt · 2023-07-10T09:24:23Z

@mreid-tt IMHO it was by intention that a manual download of deactivated packages was possible.
The former publish workflow was to manually download, install and test the packages before activation.

@hgy59, thanks for the perspective. If the download, install and test workflow was for admins then there should be a way for admins to do so via the admin portal (maybe on the build page they could be hyperlinked). If on the other hand, this was supposed to be viewable by the public then I don't see the point of having an activate function at all.

hgy59 · 2023-07-10T15:21:08Z

If on the other hand, this was supposed to be viewable by the public then I don't see the point of having an activate function at all.

Inactive packages are not visible in the package center of DSM.

mreid-tt · 2023-07-11T00:40:16Z

Inactive packages are not visible in the package center of DSM.

Really? So when we had this issue (SynoCommunity/spksrc#5798 (comment)) we could have just disabled it rather than deleting it?

hgy59 · 2023-09-11T17:34:25Z

There are still use cases that require the old behaviour.
SynoCommunity/spksrc#5684 (comment)

mreid-tt added the bug label Apr 26, 2023

This was referenced Apr 27, 2023

Implement package filter for active build #98

Merged

Template fix to show only active builds #99

Merged

mreid-tt self-assigned this Apr 27, 2023

Diaoul closed this as completed in #99 Apr 27, 2023

hgy59 mentioned this issue Jul 10, 2023

Where is the debian-chroot package for DSM5.2? SynoCommunity/spksrc#5803

Closed

hgy59 mentioned this issue Sep 11, 2023

Unable to start HomeAssistant Core on DS213+ SynoCommunity/spksrc#5684

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deactivated packages listed on website #97

Deactivated packages listed on website #97

mreid-tt commented Apr 26, 2023 •

edited

Loading

mreid-tt commented Apr 27, 2023 •

edited

Loading

mreid-tt commented Apr 27, 2023

Diaoul commented Apr 29, 2023

mreid-tt commented May 3, 2023

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023 •

edited

Loading

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023 •

edited

Loading

mreid-tt commented May 3, 2023

Diaoul commented May 4, 2023 •

edited

Loading

publicarray commented May 10, 2023

Diaoul commented May 16, 2023

th0ma7 commented May 16, 2023

mreid-tt commented Jul 10, 2023

hgy59 commented Jul 10, 2023

mreid-tt commented Jul 11, 2023

hgy59 commented Sep 11, 2023

Deactivated packages listed on website #97

Deactivated packages listed on website #97

Comments

mreid-tt commented Apr 26, 2023 • edited Loading

mreid-tt commented Apr 27, 2023 • edited Loading

mreid-tt commented Apr 27, 2023

Diaoul commented Apr 29, 2023

mreid-tt commented May 3, 2023

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023 • edited Loading

Diaoul commented May 3, 2023

mreid-tt commented May 3, 2023 • edited Loading

mreid-tt commented May 3, 2023

Diaoul commented May 4, 2023 • edited Loading

publicarray commented May 10, 2023

Diaoul commented May 16, 2023

th0ma7 commented May 16, 2023

mreid-tt commented Jul 10, 2023

hgy59 commented Jul 10, 2023

mreid-tt commented Jul 11, 2023

hgy59 commented Sep 11, 2023

mreid-tt commented Apr 26, 2023 •

edited

Loading

mreid-tt commented Apr 27, 2023 •

edited

Loading

mreid-tt commented May 3, 2023 •

edited

Loading

mreid-tt commented May 3, 2023 •

edited

Loading

Diaoul commented May 4, 2023 •

edited

Loading