UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-1: ordinal not in range(256)\n", #48
Comments
|
does your mariadb password contain any accented characters? the current default encoding will only accept the first 256 unicode characters. could you show the mariadb configuration? |
|
I used the simple [password, "mypassword" see below; ## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
# global:
# imageRegistry: myRegistryName
# imagePullSecrets:
# - myRegistryKeySecretName
# storageClass: myStorageClass
## Use an alternate scheduler, e.g. "stork".
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
# schedulerName:
## Bitnami MariaDB image
## ref: https://hub.docker.com/r/bitnami/mariadb/tags/
##
image:
registry: docker.io
repository: bitnami/mariadb
tag: 10.3.23-debian-10-r44
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistryKeySecretName
## Set to true if you would like to see extra information on logs
## It turns BASH and NAMI debugging in minideb
## ref: https://github.com/bitnami/minideb-extras/#turn-on-bash-debugging
debug: false
## String to partially override mariadb.fullname template (will maintain the release name)
##
# nameOverride:
## String to fully override mariadb.fullname template
##
# fullnameOverride:
## Additional pod labels
##
# podLabels:
# extraLabel: extraValue
## Init containers parameters:
## volumePermissions: Change the owner and group of the persistent volume mountpoint to runAsUser:fsGroup values from the securityContext section.
##
volumePermissions:
enabled: false
image:
registry: docker.io
repository: bitnami/minideb
tag: buster
pullPolicy: Always
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistryKeySecretName
resources: {}
service:
## Kubernetes service type, ClusterIP and NodePort are supported at present
type: ClusterIP
# clusterIp:
# master: xx.xx.xx.xx
# slave: xx.xx.xx.xx
port: 3306
## Specify the nodePort value for the LoadBalancer and NodePort service types.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
##
# nodePort:
# master: 30001
# slave: 30002
## Pods Service Account
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
serviceAccount:
## Specifies whether a ServiceAccount should be created
##
create: false
## The name of the ServiceAccount to use.
## If not set and create is true, a name is generated using the mariadb.fullname template
# name:
## Role Based Access
## Ref: https://kubernetes.io/docs/admin/authorization/rbac/
##
rbac:
create: false
## Pod Security Context
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
##
securityContext:
enabled: true
fsGroup: 1001
runAsUser: 1001
## Use existing secret (ignores root, db and replication passwords)
##
# existingSecret:
## MariaDB admin credentials
##
rootUser:
## MariaDB admin password
## ref: https://github.com/bitnami/bitnami-docker-mariadb#setting-the-root-password-on-first-run
##
password: mypassword
## Option to force users to specify a password. That is required for 'helm upgrade' to work properly.
## If it is not force, a random password will be generated.
##
forcePassword: true
## Mount admin password as a file instead of using an environment variable
##
injectSecretsAsVolume: true
## Custom user/db credentials
##
db:
## MariaDB username and password
## ref: https://github.com/bitnami/bitnami-docker-mariadb#creating-a-database-user-on-first-run
##
user: admin
password: mypassword
## Database to create
## ref: https://github.com/bitnami/bitnami-docker-mariadb#creating-a-database-on-first-run
##
name: erpnext_db
## Option to force users to specify a password. That is required for 'helm upgrade' to work properly.
## If it is not force, a random password will be generated.
##
forcePassword: true
## Mount user password as a file instead of using an environment variable
##
injectSecretsAsVolume: true
## Replication configuration
##
replication:
## Enable replication. This enables the creation of replicas of MariaDB. If false, only a
## master deployment would be created
##
enabled: true
## MariaDB replication user
## ref: https://github.com/bitnami/bitnami-docker-mariadb#setting-up-a-replication-cluster
##
user: replicator
## MariaDB replication user password
## ref: https://github.com/bitnami/bitnami-docker-mariadb#setting-up-a-replication-cluster
##
password: mypassword
## Option to force users to specify a password. That is required for 'helm upgrade' to work properly.
## If it is not force, a random password will be generated.
##
forcePassword: true
## Mount replication user password as a file instead of using an environment variable
##
injectSecretsAsVolume: true
## initdb scripts
## Specify dictionary of scripts to be run at first boot
## Alternatively, you can put your scripts under the files/docker-entrypoint-initdb.d directory
##
# initdbScripts:
# my_init_script.sh: |
# #!/bin/sh
# echo "Do something."
#
## ConfigMap with scripts to be run at first boot
## Note: This will override initdbScripts
# initdbScriptsConfigMap:
master:
## Mariadb Master additional pod annotations
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
# annotations:
# key: value
# another-key: another-value
## MariaDB additional command line flags
## Can be used to specify command line flags, for example:
##
## extraFlags: "--max-connect-errors=1000 --max_connections=155"
## Affinity for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
##
affinity: {}
## An array to add extra environment variables
## For example:
## extraEnvVars:
## - name: TZ
## value: "Europe/Paris"
##
extraEnvVars:
## ConfigMap with extra env vars:
##
extraEnvVarsCM:
## Secret with extra env vars:
##
extraEnvVarsSecret:
## Kept for backwards compatibility. You can now disable it by removing it.
## if you wish to set it through master.affinity.podAntiAffinity instead.
##
antiAffinity: soft
## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
## Tolerations for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
tolerations: []
## updateStrategy for MariaDB Master StatefulSet
## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
updateStrategy:
type: RollingUpdate
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
## If true, use a Persistent Volume Claim, If false, use emptyDir
##
enabled: true
# Enable persistence using an existing PVC
# existingClaim:
# Subdirectory of the volume to mount
# subPath:
mountPath: /bitnami/mariadb
## Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: "-"
## Persistent Volume Claim annotations
##
annotations: {}
## Persistent Volume Access Mode
##
accessModes:
- ReadWriteOnce
## Persistent Volume size
##
size: 8Gi
extraInitContainers: []
## An array to add extra environment variables
## For example:
## extraEnvVars:
## - name: TZ
## value: "Europe/Paris"
##
# extraEnvVars:
## Configure MySQL with a custom my.cnf file
## ref: https://mysql.com/kb/en/mysql/configuring-mysql-with-mycnf/#example-of-configuration-file
##
config: |-
[mysqld]
character-set-client-handshake=FALSE
skip-name-resolve
explicit_defaults_for_timestamp
basedir=/opt/bitnami/mariadb
plugin_dir=/opt/bitnami/mariadb/plugin
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
tmpdir=/opt/bitnami/mariadb/tmp
max_allowed_packet=16M
bind-address=0.0.0.0
pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
log-error=/opt/bitnami/mariadb/logs/mysqld.log
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
[client]
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
default-character-set=utf8mb4
plugin_dir=/opt/bitnami/mariadb/plugin
[manager]
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
## Configure master resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
livenessProbe:
enabled: true
##
## Initializing the database could take some time
initialDelaySeconds: 120
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
readinessProbe:
enabled: true
initialDelaySeconds: 30
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
podDisruptionBudget:
enabled: false
minAvailable: 1
# maxUnavailable: 1
## Allow customization of the service resource
##
service:
## Add custom annotations to the service
##
annotations: {}
slave:
replicas: 2
## Mariadb Slave additional pod annotations
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
# annotations:
# key: value
# another-key: another-value
## MariaDB additional command line flags
## Can be used to specify command line flags, for example:
##
## extraFlags: --max-connect-errors=1000 --max_connections=155"
## An array to add extra environment variables
## For example:
## extraEnvVars:
## - name: TZ
## value: "Europe/Paris"
##
extraEnvVars:
## ConfigMap with extra env vars:
##
extraEnvVarsCM:
## Secret with extra env vars:
##
extraEnvVarsSecret:
## Affinity for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
##
affinity: {}
## Kept for backwards compatibility. You can now disable it by removing it.
## if you wish to set it through slave.affinity.podAntiAffinity instead.
##
antiAffinity: soft
## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
## Tolerations for pod assignment
## Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
tolerations: []
## updateStrategy for MariaDB Slave StatefulSet
## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
updateStrategy:
type: RollingUpdate
persistence:
## If true, use a Persistent Volume Claim, If false, use emptyDir
##
enabled: true
# storageClass: "-"
annotations:
accessModes:
- ReadWriteOnce
## Persistent Volume size
##
size: 8Gi
## Add extra init containers
##
extraInitContainers: []
## An array to add extra environment variables
## For example:
## extraEnvVars:
## - name: TZ
## value: "Europe/Paris"
##
# extraEnvVars:
## Configure MySQL slave with a custom my.cnf file
## ref: https://mysql.com/kb/en/mysql/configuring-mysql-with-mycnf/#example-of-configuration-file
##
config: |-
[mysqld]
character-set-client-handshake=FALSE
skip-name-resolve
explicit_defaults_for_timestamp
basedir=/opt/bitnami/mariadb
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
tmpdir=/opt/bitnami/mariadb/tmp
max_allowed_packet=16M
bind-address=0.0.0.0
pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
log-error=/opt/bitnami/mariadb/logs/mysqld.log
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
[client]
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
default-character-set=utf8mb4
[manager]
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
##
## Configure slave resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
livenessProbe:
enabled: true
##
## Initializing the database could take some time
initialDelaySeconds: 120
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
readinessProbe:
enabled: true
initialDelaySeconds: 45
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
podDisruptionBudget:
enabled: false
minAvailable: 1
# maxUnavailable: 1
## Allow customization of the service resource
##
service:
## Add custom annotations to the service
##
annotations: {}
metrics:
enabled: true
image:
registry: docker.io
repository: bitnami/mysqld-exporter
tag: 0.12.1-debian-10-r146
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistryKeySecretName
resources: {}
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9104"
## Extra args to be passed to mysqld_exporter
## ref: https://github.com/prometheus/mysqld_exporter/
##
extraArgs:
master: []
slave: []
# - --collect.auto_increment.columns
# - --collect.binlog_size
# - --collect.engine_innodb_status
# - --collect.engine_tokudb_status
# - --collect.global_status
# - --collect.global_variables
# - --collect.info_schema.clientstats
# - --collect.info_schema.innodb_metrics
# - --collect.info_schema.innodb_tablespaces
# - --collect.info_schema.innodb_cmp
# - --collect.info_schema.innodb_cmpmem
# - --collect.info_schema.processlist
# - --collect.info_schema.processlist.min_time
# - --collect.info_schema.query_response_time
# - --collect.info_schema.tables
# - --collect.info_schema.tables.databases
# - --collect.info_schema.tablestats
# - --collect.info_schema.userstats
# - --collect.perf_schema.eventsstatements
# - --collect.perf_schema.eventsstatements.digest_text_limit
# - --collect.perf_schema.eventsstatements.limit
# - --collect.perf_schema.eventsstatements.timelimit
# - --collect.perf_schema.eventswaits
# - --collect.perf_schema.file_events
# - --collect.perf_schema.file_instances
# - --collect.perf_schema.indexiowaits
# - --collect.perf_schema.tableiowaits
# - --collect.perf_schema.tablelocks
# - --collect.perf_schema.replication_group_member_stats
# - --collect.slave_status
# - --collect.slave_hosts
# - --collect.heartbeat
# - --collect.heartbeat.database
# - --collect.heartbeat.table
livenessProbe:
enabled: true
##
## Initializing the database could take some time
initialDelaySeconds: 120
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
readinessProbe:
enabled: true
initialDelaySeconds: 30
##
## Default Kubernetes values
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
# Enable this if you're using https://github.com/coreos/prometheus-operator
serviceMonitor:
enabled: false
## Specify a namespace if needed
# namespace: monitoring
# fallback to the prometheus default unless specified
# interval: 10s
# scrapeTimeout: 10s
## Defaults to what's used if you follow CoreOS [Prometheus Install Instructions](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#tldr)
## [Prometheus Selector Label](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#prometheus-operator-1)
## [Kube Prometheus Selector Label](https://github.com/bitnami/charts/tree/master/bitnami/prometheus-operator#exporters)
selector:
prometheus: kube-prometheus
## Bats Framework (= Bash Automated Testing System) is needed to test if MariaDB is accessible
## See test-runner.yaml and tests.yaml for details.
## To run the tests after the deployment, enter "helm test <release-name>".
tests:
enabled: true
resources: {}
testFramework:
image:
registry: docker.io
repository: dduportal/bats
tag: 0.4.0
resources: {} |
|
The db password is stored as a secret in k8s. So I suspected that the encryption process could have added extra characters to the password kubectl apply -n erpnext -f - <<EOF |
|
can you get the secret and see what it says: kubectl get secret --namespace mariadb mariadb -o jsonpath="{.data.mariadb-root-password}" |
|
I get this below enRZdW5YekJIRHox |
|
did you encode the password to base64 before creating the secret? refer: https://helm.erpnext.com/kubernetes-resources/create-mariadb-root-password-secret |
|
No I did not. Let me try with an encoded password |
|
I used the encoded password and now I get the error below; E 2020-07-23T07:32:45.305990106Z E 2020-07-23T07:32:45.305995222Z Traceback (most recent call last): undefined |
|
could you send me the output again: kubectl get secret --namespace mariadb mariadb -o jsonpath="{.data.mariadb-root-password}" |
|
did you make the same changes to the mariadb chart as well? you will have to do so, reapply the chart, and then retry new-site |
|
the chart should contain the unhashed password, and the secrets should have the base64 hash of the password. |
|
I did just that exactly. |
|
I have tried again and still same results. |
|
What are the StorageClass being used?
here is my list of PVCs. (scw-bssd is provided by scaleway, google will have their own) Can you manually connect to mariadb with instructions from |
|
$ kubectl get pvc --all-namespaces NAMESPACE NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE I can manually connect to both Master and Slave , see below; kubectl run mariadb-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mariadb:10.3.23-debian-10-r44 --namespace mariadb --command -- bash Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [erpnext_db]> I have no name!@mariadb-client:/$ mysql -h mariadb-slave.mariadb.svc.cluster.local -uroot -p erpnext_db Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [erpnext_db]> |
|
If you can login manually then password in erpnext namespace must be wrong. compare passwords |
|
Thank you. Here is how I am applying the mariadb secret - kubectl apply -n erpnext -f - <<EOF Is there anything wrong with the config? cGFzc3dvcmQ= is the encoded Base64 password |
|
I updated the compare passwords command, there was a mistake. I created environment variable MARIADB_ERPNEXT_SECRET and compared MARIADB_SECRET updated command in previous comment. to get BASE64_PASSWORD mentioned here https://helm.erpnext.com/kubernetes-resources/create-mariadb-root-password-secret |
|
The passwords are matching |
|
BASE64_PASSWORD=$(kubectl get secret --namespace mariadb mariadb -o jsonpath="{.data.mariadb-root-password}") |
|
Everything seems to be correct on your end. even tests are passing https://travis-ci.com/github/frappe/frappe_docker/jobs/363864114 I have not tried anything on GKE yet. I'll try if there is any free tier available. |
|
Yes, the DB issues have been resolved. |
Description of the issue
I am installing ERPNext on GKE using the Helm Chart. I followed the steps here - https://helm.erpnext.com/faq.html - and when I get to the step - Create New Site Job. I get the error - UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-1: ordinal not in range(256)\n",
Context information (for bug reports)
I follow the steps outlined successfully until I have to create a new site using the config below;
kubectl apply -n erpnext -f - <<EOF
apiVersion: batch/v1
kind: Job
metadata:
name: create-new-site
spec:
backoffLimit: 1
template:
spec:
securityContext:
supplementalGroups: [1000]
containers:
- name: create-site
image: frappe/erpnext-worker:v12.10.1
args: ["new"]
imagePullPolicy: IfNotPresent
volumeMounts:
- name: sites-dir
mountPath: /home/frappe/frappe-bench/sites
env:
- name: "SITE_NAME"
value: staging.example.com
- name: "DB_ROOT_USER"
value: root
- name: "MYSQL_ROOT_PASSWORD"
valueFrom:
secretKeyRef:
key: password
name: mariadb-root-password
- name: "ADMIN_PASSWORD"
value: mypassword
- name: "INSTALL_APPS"
value: "erpnext"
restartPolicy: Never
volumes:
- name: sites-dir
persistentVolumeClaim:
claimName: frappe-bench-0001-erpnext
readOnly: false
EOF
Steps to reproduce the issue
1.I run the command above to create a new site and then I navigate to my google cloud console to check
2.Under Kubernetes Engine > Workload I notice the Status for the Create New Site Job says BackoffLimit Exceeded
3.I then click on that particular Job > Job Details > Cloud logs and I see the Observed result below
Observed result
Traceback (most recent call last): File "/home/frappe/frappe-bench/commands/new.py", line 106, in main() File "/home/frappe/frappe-bench/commands/new.py", line 59, in main db_port=db_port, File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 89, in _new_site db_password=db_password, db_type=db_type, db_host=db_host, db_port=db_port, no_mariadb_socket=no_mariadb_socket) File "/home/frappe/frappe-bench/apps/frappe/frappe/installer.py", line 35, in install_db setup_database(force, source_sql, verbose, no_mariadb_socket) File "/home/frappe/frappe-bench/apps/frappe/frappe/database/init.py", line 16, in setup_database return frappe.database.mariadb.setup_db.setup_database(force, source_sql, verbose, no_mariadb_socket=no_mariadb_socket) File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/setup_db.py", line 39, in setup_database if force or (db_name not in dbman.get_database_list()): File "/home/frappe/frappe-bench/apps/frappe/frappe/database/db_manager.py", line 61, in get_database_list return [d[0] for d in self.db.sql("SHOW DATABASES")] File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 122, in sql self.connect() File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 75, in connect self._conn = self.get_connection() File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/database.py", line 90, in get_connection local_infile = frappe.conf.local_infile) File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/init.py", line 94, in Connect return Connection(*args, **kwargs) File "/home/frappe/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py", line 260, in init self.password = self.password.encode('latin1') UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-1: ordinal not in range(256)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-1: ordinal not in range(256)
"
timestamp: "2020-07-22T18:38:34.283260158Z"
}
2020-07-22 20:38:57.482 SAST
Attempt 1 to connect to mariadb.mariadb.svc.cluster.local:3306
2020-07-22 20:38:57.515 SAST
Attempt 1 to connect to frappe-bench-0001-erpnext-redis-queue:12000
2020-07-22 20:38:57.522 SAST
Attempt 1 to connect to frappe-bench-0001-erpnext-redis-cache:13000
2020-07-22 20:38:57.530 SAST
Attempt 1 to connect to frappe-bench-0001-erpnext-redis-socketio:11000
2020-07-22 20:38:57.697 SAST
Connections OK
2020-07-22 20:39:17.495 SAST
Site staging.example.com already exists
Expected result
I expected a green OK Status
Stacktrace / full error message if available
The text was updated successfully, but these errors were encountered: