Skip to content

Commit

Permalink
Merge branch 'master' into enable-fts5
Browse files Browse the repository at this point in the history
# Conflicts:
#	demo/addons/godot-sqlite/bin/android/arm64-v8a/libgdsqlite.so
#	demo/addons/godot-sqlite/bin/android/armeabi-v7a/libgdsqlite.so
#	demo/addons/godot-sqlite/bin/android/x86/libgdsqlite.so
#	demo/addons/godot-sqlite/bin/ios/arm64/libgdsqlite.a
#	demo/addons/godot-sqlite/bin/ios/armv7/libgdsqlite.a
#	demo/addons/godot-sqlite/bin/osx/libgdsqlite.dylib
#	demo/addons/godot-sqlite/bin/win64/libgdsqlite.dll
#	demo/addons/godot-sqlite/bin/x11/libgdsqlite.so
  • Loading branch information
2shady4u committed Apr 4, 2021
2 parents 6ddc02e + 33439ea commit 4f8e758
Show file tree
Hide file tree
Showing 17 changed files with 8,188 additions and 5,101 deletions.
53 changes: 51 additions & 2 deletions .github/workflows/linux_builds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ env:

jobs:
linux-compilation:
name: Linux Compilation
name: Ubuntu 20.04 Compilation
runs-on: "ubuntu-20.04"
steps:
- name: Checkout
Expand Down Expand Up @@ -59,4 +59,53 @@ jobs:
uses: actions/upload-artifact@v1
with:
name: x11
path: ${{env.PROJECT_FOLDER}}/${{env.TARGET_PATH}}
path: ${{env.PROJECT_FOLDER}}/${{env.TARGET_PATH}}
linux-legacy-compilation:
name: Ubuntu 16.04 Compilation
runs-on: "ubuntu-16.04"
steps:
- name: Checkout
uses: actions/checkout@v2
with:
lfs: true
submodules: recursive

# Install all packages (except scons)
- name: Configure dependencies
run: |
sudo apt-get update
sudo apt-get install build-essential pkg-config libx11-dev libxcursor-dev \
libxinerama-dev libgl1-mesa-dev libglu-dev libasound2-dev libpulse-dev libudev-dev libxi-dev libxrandr-dev yasm
# Use python 3.x release (works cross platform; best to keep self contained in it's own step)
- name: Set up Python 3.x
uses: actions/setup-python@v2
with:
# Semantic version range syntax or exact version of a Python version
python-version: '3.x'
# Optional - x64 or x86 architecture, defaults to x64
architecture: 'x64'

# Setup scons, print python version and scons version info, so if anything is broken it won't run the build.
- name: Configuring Python packages
run: |
python -c "import sys; print(sys.version)"
python -m pip install scons
python --version
scons --version
# Linux shared libraries are humongous (>70MB) for some reason.. don't forget to strip them!
- name: Compilation
run: |
mkdir -v -p ${{env.PROJECT_FOLDER}}/${{env.TARGET_PATH}}
cd ${{env.PROJECT_FOLDER}}
cd godot-cpp
scons platform=linux bits=64 target=${{env.TARGET}} generate_bindings=yes -j4
cd ..
scons platform=linux target=${{env.TARGET}} target_path=${{env.TARGET_PATH}} target_name=${{env.TARGET_NAME}}
- name: Upload Artifact
uses: actions/upload-artifact@v1
with:
name: x11-legacy
path: ${{env.PROJECT_FOLDER}}/${{env.TARGET_PATH}}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Thumbs.db
.sconsign.dblite
demo/build
test_backup_new.json
test_backup_base64_new.json

# VS
vsproj/.vs/
Expand Down
2 changes: 1 addition & 1 deletion Android.mk
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ src/library.cpp \
src/sqlite/sqlite3.c \

LOCAL_C_INCLUDES := \
godot-cpp/godot_headers \
godot-cpp/godot-headers \
godot-cpp/include/ \
godot-cpp/include/core \
godot-cpp/include/gen \
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ In chronological order based on the first contribution.
- [flomar](https://github.com/flomar)
- [Aaron Franke (aaronfranke)](https://github.com/aaronfranke)
- [Akshay Sunil Masare (aksmas)](https://github.com/aksmas)
- [Geoffrey Casper (Geo25rey)](https://github.com/Geo25rey)
99 changes: 84 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ Enables or disables the availability of [foreign keys](https://www.sqlite.org/fo

- **query_result** (Array, default=[])

Contains the results from the latest query and is cleared after every new query.
Contains the results from the latest query and is cleared after every new query.

***NOTE:** If you want your result to persist you'll have to **duplicate()** this array yourself BEFORE running additional queries.*

- **last_insert_rowid** (Integer, default=0)

Expand Down Expand Up @@ -87,6 +89,8 @@ var success = db.query_with_bindings(query_string, param_bindings)

Using bindings is optional, except for PoolByteArray (= raw binary data) which has to binded to allow the insertion and selection of BLOB data in the database.

***NOTE**: Binding column names is not possible due SQLite restrictions. If dynamic column names are required, insert the column name directly into the `query_string`-variable itself (see https://github.com/2shady4u/godot-sqlite/issues/41).*

- Boolean success = **create_table(** String table_name, Dictionary table_dictionary **)**

Each key/value pair of the `table_dictionary`-variable defines a column of the table. Each key defines the name of a column in the database, while the value is a dictionary that contains further column specifications.
Expand Down Expand Up @@ -173,39 +177,104 @@ Bind a [scalar SQL function](https://www.sqlite.org/appfunc.html) to the databas

There are a couple of things you can do before panicking, namely:
- Test out if your query is valid by trying it out online at https://sqliteonline.com/.
- Encapsulate all conditional statements in tick-marks ', for example:
- Use the **query(** **)** or **query_with_bindings(** **)**-function instead of the more specialized wrapper function.
- Your query might be missing some quotation marks. For example, following queries will fail due to missing encapsulation of the `default`-field:

```gdscript
var table_name := "characters"
var table_dict : Dictionary
table_dict["last_name"] = {"data_type":"text", "default": "Silver"}
table_dict["first_name"] = {"data_type":"text", "default": "Long John"}
table_dict["area"] = {"data_type":"text", "default": ""}
table_dict["color"] = {"data_type":"text", "default": "0,0,0,0"}
db.create_table(table_name, table_dict)
```

Adding some well-placed single quotation marks fixes this issue:

```gdscript
var table_name := "characters"
var table_dict : Dictionary
table_dict["last_name"] = {"data_type":"text", "default": "Silver"}
table_dict["first_name"] = {"data_type":"text", "default": "'Long John'"}
table_dict["area"] = {"data_type":"text", "default": "''"}
table_dict["color"] = {"data_type":"text", "default": "'0,0,0,0'"}
db.create_table(table_name, table_dict)
```

Basically you'll need to use single quotation marks whenever:
- The string is empty
- The string contains syntax restricted symbols such as commas or spaces

- SQLite restricts dynamically binding the names of tables and columns, thus following query will fail due to syntax errors:
```gdscript
var table_name := "characters"
var column_name := "level"
db.query_with_bindings("UPDATE ? SET ?=? WHERE id=?", [table_name, column_name, 100, 1])
```

This is forbidden SQLite syntax as both the `table_name`- and `column_name`-variables cannot be bound! If dynamic modification of names of tables and columns is required for purposes of your code, then use following work-around:

```gdscript
var table_name := "characters"
var column_name := "level"
db.query_with_bindings("UPDATE "+ table_name +" SET "+ column_name +"=? WHERE id=?", [100, 1])
```

After exhausting these options, please open an issue that describes the error in proper detail.

### 2. Your plugin fails to load on my Windows machine!

Basically if your Windows machine device doesn't have the required VC++ redistributables installed, the dynamic library will fail to load and throw an error of the following sort:

The following statement might not work and throw syntax errors:
```swift
var rows = db.select_rows(table_name, "number >= {0} AND number < {0} + {1}".format([i, step]), ["*"])
```

Encapsulating the conditonal statements with tick-marks ' fixes these syntax errors:
```swift
var rows = db.select_rows(table_name, "number >= {0} AND number < {0} + {1}".format([i, step]), ["*"])
ERROR: GDNative::get_symbol: No valid library handle, can't get symbol from GDNative object
At: modules\gdnative\gdnative.cpp:315
ERROR: NativeScriptLanguage::init_library: No nativescript_init in "res://addons/godot-sqlite/bin/win64/libgdsqlite.dll" found
At: modules\gdnative\nativescript\nativescript.cpp:1054
```
This issue is still open and is being actively worked on.

- Using the **query(** **)**-function instead of the more specialized wrapper function.
This is an open issue that is still under consideration (see https://github.com/2shady4u/godot-sqlite/issues/33).

Some possible solutions/work-arounds exist:
- Install the missing VC++ redistributables (downloadable [here](https://support.microsoft.com/en-us/topic/the-latest-supported-visual-c-downloads-2647da03-1eea-4433-9aff-95f26a218cc0))
- Recompile the plugin using the MinGW compiler instead (**WARNING:** currently results in a >15MB library).
- Recompile the plugin (and the bindings) using the `/MT`-flag instead of the `/MD`-flag as discussed [here](https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-160).

After exhausting these options, please open an issue that describes the error in detail.
If the console error is of an entirely different nature, please open an issue.

### 2. When should I create function bindings to augment SQLite's set of native functions?
### 3. When should I create function bindings to augment SQLite's set of native functions?

Preferably never.

Creating function should only be seen as a measure of last resort and only be used when you perfectly know what you are doing. Be sure to first check out the available native list of [scalar SQL applications](https://www.sqlite.org/lang_corefunc.html) that is already available in SQLite3.

### 3. My Android (or iOS) application cannot access the database!
### 4. My Android (or iOS) application cannot access the database!

Android does not allow modification of files in the 'res://'-folder, thus blocking the plugin from writing to and/or reading from this database-file.
In both cases, the most painless solution is to copy the entire database to the 'user://-folder' as apps have explicit writing privileges there.

If there is a better solution, one that does not involve copying the database to a new location, please do enlighten me.

### 5. Is this plugin compatible with a Godot Server binary? How to set it up?

This plugin is fully compatible with the Godot Server binary.
Follow these steps to create a working Linux Server for your project:

1. Export your project's `*.pck` using Godot's export functionalities for Linux.
2. Alongside the exported package, paste the following files:
- `libgdsqlite.so` (as found in `addons/godot-sqlite/bin/x11/`)
- Your project's database(s) (`*.db`)
- The Godot Server binary as downloadable [here](https://godotengine.org/download/server)
3. Rename the Godot Server binary to have the exact same name as the exported `*.pck`
(for example if your package is called `game.pck`, your binary should be named `game.x64`)
4. Done!

***NOTE**: If you are using an older version of Linux on your server machine (with glibc version < 2.28), the plugin crashes due to the compiled version of glibc being too recent. In that case you can either recompile the Linux plugin binary yourself or you can download the legacy binaries (Ubuntu 16.04 with glibc version == 2.23) as found [here](https://github.com/2shady4u/godot-sqlite/actions/workflows/linux_builds.yml).*

# How to export?

**NOTE**: On mobile platforms (Android & iOS) this is not possible and the 'res://data/'-folder has to be copied to the 'user://-folder' in its entirety instead (see FAQ above).
***NOTE**: On mobile platforms (Android & iOS) the method discussed here is not possible and the contents of the `res://data/`-folder has to be copied to the `user://-folder` in its entirety instead (see FAQ above).*

All json- and db-files should be part of the exact same folder (demo/data in the case of the demo-project). During export this folder should be copied in its entirety to the demo/build-folder, in which the executable will be created by Godot's export command line utilities. Luckily, a Godot script called 'export_data.gd' can also found in the demo-project and allows to automatically copy the demo/data-folder's contents to the demo/build-folder.

Expand Down
2 changes: 1 addition & 1 deletion SConstruct
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ opts.Add(PathVariable(
))

# Local dependency paths, adapt them to your setup
godot_headers_path = "godot-cpp/godot_headers/"
godot_headers_path = "godot-cpp/godot-headers/"
cpp_bindings_path = "godot-cpp/"

env = Environment(ENV = os.environ)
Expand Down
2 changes: 1 addition & 1 deletion demo/addons/godot-sqlite/plugin.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
name="Godot SQLite"
description="GDNative wrapper for SQLite (Godot 3.1+), making it possible to use SQLite databases as data storage in all your future games."
author="Piet Bronders & Jeroen De Geeter"
version="2.2"
version="2.3"
script="godot-sqlite.gd"
14 changes: 14 additions & 0 deletions demo/data/test_backup_base64_old.json

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions demo/data/test_backup_old.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[
{
"name": "company",
"type": "table",
"sql": "CREATE TABLE company (id int PRIMARY KEY NOT NULL,name text NOT NULL,age int NOT NULL,address char(50),salary real)",
"row_array": [
{
Expand Down Expand Up @@ -42,6 +43,7 @@
},
{
"name": "office_supply",
"type": "table",
"sql": "CREATE TABLE office_supply (id int PRIMARY KEY NOT NULL,name text NOT NULL,amount int NOT NULL)",
"row_array": [
{
Expand Down
17 changes: 17 additions & 0 deletions demo/database.gd
Original file line number Diff line number Diff line change
Expand Up @@ -64,5 +64,22 @@ func example_of_fts5_usage():
db.query("SELECT * FROM " + table_name + " WHERE posts MATCH 'learn SQLite';")
cprint("result: {0}".format([String(db.query_result)]))

# Export the table to a json-file and automatically encode BLOB data to base64.
db.export_to_json(json_name + "_base64_new")

# Import again!
db.import_from_json(json_name + "_base64_old")

# Check out the 'old' icon stored in this backup file!
selected_array = db.select_rows(table_name, "", ["data"])
for selected_row in selected_array:
var selected_data = selected_row.get("data", PoolByteArray())

var image := Image.new()
var _error : int = image.load_png_from_buffer(selected_data)
var loaded_texture := ImageTexture.new()
loaded_texture.create_from_image(image)
emit_signal("texture_received", loaded_texture)

# Close the current database
db.close_db()
2 changes: 1 addition & 1 deletion godot-cpp
Loading

0 comments on commit 4f8e758

Please sign in to comment.