Many functionalities missing for vertex/edge collections

[akki]

There are many functionalities missing for the vertex/edge collections that are present for collections created by normal create_collection method of the database.
Some of them are to allow creating shards, allowing bulk_import.

[joowani]

Hi @akki

You can access the vertex/edge collection using the standard collection interface like db.col('my_vertex_collection_name'). Let me know if that meets your needs.

[akki]

Thanks for the quick reply @joowani
I am currently doing db.col('my_vertex_collection_name') as an alternative. My main concern was regarding creating shards. I am unable to create shards for vertex/edge collections. I was kinda under the impression that vertex/edge collections would be subclasses of standard collections but when I saw the code that wasn't the case.

[joowani]

Hi @akki,

Right. The reason for the not-very-clean separation is due to the limitation of the HTTP REST API provided by ArangoDB (which python-arango uses under the hood). For now, you will have to create the collections you need for your graph using the standard collections interface so you can specify the shard configuration, and then create graph by using those collections and its vertices.

[akki]

Yes, I create my collections using Arangosh currently.
Is it possible to do it in some way using multi-inheritance (I haven't looked at the code very closely right now).
Anyways, this isn't a very critical issue and there are work-arounds but I thought of reporting it. Thanks for the help!

[joowani]

Noted. I'll see what can be done to address this next release. Thanks @akki!

[jeeveeyes]

Hi @joowani , Thanks for creating this library to make Arango easy to access from Python.

Please see arangodb/arangodb#3020 that I had raised in ArangoDB.

As per the suggestion there ,I am supposed to use replication factor while creating vertex collections. But there is no API to do that in python-arango for vertex-collections or at graph level itself.

Do you have any suggestions how I could achieve that or is an enhancement needed in python-arango to support that ?

[joowani]

Hi @jeeveeyes,

In the latest version of python-arango, there is actually a way to set the replication factor by using Database.create_collection. You can create the collection using this method instead of Graph.create_vertex_collection (which is pretty much the same thing without the options) and assign it to your graph's edge definition:

from arango import ArangoClient

client = ArangoClient()
db = client.db('my_database')
graph = db.create_graph('my_graph')
db.create_collection('my_col', replication_factor=2)

graph.create_edge_definition(
    name='my_def',
    from_collections=['my_col'],
    to_collections=['my_col']
)

Best,
Joohwan

[jeeveeyes]

Thanks @joowani , will try the workaround that you suggested.

[jeeveeyes]

@joowani, the db.create_collection() seems to set the replication_factor correctly for vertex collections.

However, for edge collections, the below method does not seem to set it properly.

graph.create_edge_definition(
    name='my_def',
    from_collections=['my_col'],
    to_collections=['my_col']
)

As per recent ArangoDB changelog

v3.2.6 (2017-10-26)
-------------------

* allow changing of collection replication factor for existing collections

* UI: replicationFactor of a collection is now changeable in a cluster
environment

Will python-arango be enhanced with more support for configuring replication factor beyond db.create_collection() ?

[kunalak]

@jeeveeyes I sent you the below snippet in earlier emails for changing the replication factor for edges and that would also be quite fast using the document API directly.

edge_colls = {}
for from_coll,to_coll in edge_data:
name = from_coll + "ref" + to_coll
db.create_collection(name,edge=True,replication_factor=2)
edge_colls[name] = grph.create_edge_definition(name=name, from_collections = [from_coll], to_collections = [to_coll])

[jeeveeyes]

Thanks @kunalak. I shall try above method.

Can you also clarify if replication factor is required to be tuned for graphs too ?

As can be seen above , for a simple graph, the default replication_factor is 1.
I could not locate the Arango REST API to set that replication_factor for a graph while creating it or as a PUT of the properties.

[jeeveeyes]

@joowani , there is a configure() API on collection classes, but that does not accept and propagate replication_factor.

[joowani]

@jeeveeyes, the configure method is just a wrapper on top of this HTTP API call: https://docs.arangodb.com/3.2/HTTP/Collection/Modifying.html#change-properties-of-a-collection.

And it does not accept replication factor as a parameter. In fact I don't know if there is one exposed by ArangoDB at all (there does not seem to be one in their HTTP API spec. For now, you may need to use something like Arangosh or the web UI directly. @akki please refer to ArangoDB documentation or create an issue with them for more details on how to do this.