Add sources.

HISTORY.rst

@@ -2,17 +2,25 @@
 ------------------
 
 * Update to skosprovider 0.6.0
-* Due to the update to skosprovider 0.6.0, a new field `markup`, was added to a 
+* Due to the update to skosprovider 0.6.0, a new field `markup`, was added to a
   :class:`skosprovider_sqlalchemy.models.Note`. When upgrading from a previous
   version of `skosprovider_sqlalchemy`, any databases created in that previous
-  verions will need to be updated as well. Please add a field called `markup` 
+  verions will need to be updated as well. Please add a field called `markup`
   to the `note` table.
 * Inline with the skosprovider 0.6.0 update, a `languages` attribute was added
-  to :class:`skosprovider_sqlalchemy.models.ConceptScheme`. When upgrading from 
-  a previous version of `skosprovider_sqlalchemy`, any databases created with 
-  that previous verions will need to be updated as well. Please add a table 
+  to :class:`skosprovider_sqlalchemy.models.ConceptScheme`. When upgrading from
+  a previous version of `skosprovider_sqlalchemy`, any databases created with
+  that previous verions will need to be updated as well. Please add a table
   called `conceptscheme_language` with fields `conceptscheme_id` and
   `language_id`. (#18)
+* To comply with the skosprovider 0.6.0 update, the `sources` attribute was
+  added to :class:`skosprovider_sqlalchemy.models.Conceptscheme`,
+  :class:`skosprovider_sqlalchemy.models.Concept` and :class:`skosprovider_sqlalchemy.models.Collection`.
+  When upgrading from a previous version of `skosprovider_sqlalchemy`, any
+  databases created with that previous verions will need to be updated as well.
+  Please add a table `source` with fields `id` and `citation`, a table
+  `concept_source` with fields `concept_id` and `source_id` and a table
+  `conceptscheme_source` with fields `conceptscheme_id` and `source_id`.
 * When importing a provider, check if the languages that are being used in the
   provider are already in our database. If not, validate them and add them to
   the database. In the pastthe entire import would fail if not all languages had
@@ -26,7 +34,7 @@
 * Make README work better on pypi.
 * Fix a further problem with the length of language identifiers. Previous fix
   in 0.3.0 only fixed the length of the identifiers in the languages table,
-  but not in the links from the labels and the notes to the language table. 
+  but not in the links from the labels and the notes to the language table.
   [BartSaelen]
 * Added some documentation about setting up a database.
 
@@ -46,23 +54,23 @@
   a session maker first and then passed the result of this call to the provider.
   Now you should just pass the session maker itself and let the provider create
   the sessions for you.
-* Different way of fetching the :class:`~skosprovider.skos.ConceptScheme` 
-  for a provider. No longer fetches a conceptscheme at provider instantiation, 
-  but when needed. Otherwise we end up with a possibly very long cached version 
+* Different way of fetching the :class:`~skosprovider.skos.ConceptScheme`
+  for a provider. No longer fetches a conceptscheme at provider instantiation,
+  but when needed. Otherwise we end up with a possibly very long cached version
   of a conceptscheme.
 
 0.3.0 (2014-10-17)
 ------------------
 
 * Update to skosprovider 0.4.0.
 * Add :class:`~skosprovider.skos.ConceptScheme` information to a provider so it
-  can be attached to :class:`~skosprovider.skos.Concept` objects that are 
+  can be attached to :class:`~skosprovider.skos.Concept` objects that are
   handled by the provider.
 * Let provider handle superordinates and subordinate arrays.
 * Let provider add notes to collections.
 * Added a :class:`~skosprovider_sqlalchemy.models.Match` model to handle
   matches. Expand the provider to actually provide information on these matches.
-* Expand the field length for language identifiers. IANA suggests that 
+* Expand the field length for language identifiers. IANA suggests that
   identifiers up to 35 characters should be permitted. Updated our field length
   to 64 to have a bit of an extra buffer.
 
@@ -74,9 +82,9 @@
 * Add ability to configure the SQLAlchemy URL used for testing. Allows testing
   on multiple RDBMS systems.
 * Run `Travis <https://travis-ci.org>`_ tests for both SQLite and Postgresql.
-* Fix a bug in :meth:`skosprovider_sqlalchemy.utils.import_provider` when 
+* Fix a bug in :meth:`skosprovider_sqlalchemy.utils.import_provider` when
   dealing with narrower collections (#8). [cahytinne]
-* Make the provider actually generate a :term:`URI` if there's none in the 
+* Make the provider actually generate a :term:`URI` if there's none in the
   database.
 
 0.2.0 (2014-05-14)
@@ -95,7 +103,7 @@
 ------------------
 
 * Pinned dependency on skosprovider < 0.3.0
-* Pass data to :class:`skosprovider.skos.Concept` using keywords in stead of 
+* Pass data to :class:`skosprovider.skos.Concept` using keywords in stead of
   positions.
 
 0.1.1 (2013-11-28)
@@ -109,7 +117,7 @@
 
 * Initial version
 * Implementation of a SKOS domain model in SQLAlchemy.
-* Implementation of a :class:`skosprovider.providers.VocabularyProvider` that 
+* Implementation of a :class:`skosprovider.providers.VocabularyProvider` that
   uses this model.
 * Can query a hierarchy recursively or using nested sets.
 * Utility function to import a :class:`skosprovider.providers.VocabularyProvider`

skosprovider_sqlalchemy/models.py

@@ -54,6 +54,13 @@
     Column('note_id', Integer, ForeignKey('note.id'), primary_key=True)
 )
 
+concept_source = Table(
+    'concept_source',
+    Base.metadata,
+    Column('concept_id', Integer, ForeignKey('concept.id'), primary_key=True),
+    Column('source_id', Integer, ForeignKey('source.id'), primary_key=True)
+)
+
 conceptscheme_note = Table(
     'conceptscheme_note',
     Base.metadata,
@@ -66,6 +73,23 @@
     Column('note_id', Integer, ForeignKey('note.id'), primary_key=True)
 )
 
+conceptscheme_source = Table(
+    'conceptscheme_source',
+    Base.metadata,
+    Column(
+        'conceptscheme_id',
+        Integer,
+        ForeignKey('conceptscheme.id'),
+        primary_key=True
+    ),
+    Column(
+        'source_id',
+        Integer,
+        ForeignKey('source.id'),
+        primary_key=True
+    )
+)
+
 conceptscheme_language = Table(
     'conceptscheme_language',
     Base.metadata,
@@ -177,6 +201,13 @@ class Thing(Base):
         cascade='all, delete-orphan',
         single_parent=True
     )
+    sources = relationship(
+        'Source',
+        secondary=concept_source,
+        backref=backref('concept', uselist=False),
+        cascade='all, delete-orphan',
+        single_parent=True
+    )
 
     conceptscheme = relationship('ConceptScheme', backref='concepts')
     conceptscheme_id = Column(
@@ -315,6 +346,13 @@ class ConceptScheme(Base):
         'Language',
         secondary=conceptscheme_language
     )
+    sources = relationship(
+        'Source',
+        secondary=conceptscheme_source,
+        backref=backref('conceptscheme', uselist=False),
+        cascade='all, delete-orphan',
+        single_parent=True
+    )
     def label(self, language='any'):
         return label(self.labels, language)
 
@@ -444,6 +482,24 @@ def __str__(self):
         return self.note
 
 
+class Source(Base):
+    '''
+    The source where a certain piece of information came from.
+    '''
+    __tablename__ = 'source'
+    id = Column(Integer, primary_key=True)
+    citation = Column(
+        Text,
+        nullable=False
+    )
+
+    def __init__(self, citation):
+        self.citation = citation
+
+    def __str__(self):
+        return self.citation
+
+
 class MatchType(Base):
     '''
     A matchType according to :term:`skosprovider:SKOS`.
@@ -463,7 +519,7 @@ def __str__(self):
 
 class Match(Base):
     '''
-    A match between a :class:`Concept` in one ConceptScheme and those in 
+    A match between a :class:`Concept` in one ConceptScheme and those in
     another one.
     '''
     __tablename__ = 'match'
@@ -540,8 +596,8 @@ def label(labels=[], language='any'):
     if no exact match is present, an inexact match will be attempted. This might
     be because a label in language `nl-BE` is being requested, but only `nl` or
     even `nl-NL` is present. Similarly, when requesting `nl`, a label with
-    language `nl-NL` or even `nl-Latn-NL` will also be considered, 
-    providing no label is present that has an exact match with the 
+    language `nl-NL` or even `nl-Latn-NL` will also be considered,
+    providing no label is present that has an exact match with the
     requested language.
 
     If language 'any' was specified, all labels will be considered,
@@ -555,7 +611,7 @@ def label(labels=[], language='any'):
     Finally, if no label could be found, None is returned.
 
     :param list labels: A list of :class:`labels <Label>`.
-    :param str language: The language for which a label should preferentially 
+    :param str language: The language for which a label should preferentially
         be returned. This should be a valid IANA language tag.
     :rtype: A :class:`Label` or `None` if no label could be found.
     '''

skosprovider_sqlalchemy/providers.py

@@ -14,7 +14,8 @@
     Concept,
     Collection,
     Label,
-    Note
+    Note,
+    Source
 )
 
 from skosprovider_sqlalchemy.models import (
@@ -119,6 +120,9 @@ def _get_concept_scheme(self):
             ],
             languages=[
                 l.id for l in csm.languages
+            ],
+            sources=[
+                Source(s.citation) for s in csm.sources
             ]
         )
 
@@ -142,6 +146,9 @@ def _from_thing(self, thing):
                     Note(n.note, n.notetype_id, n.language_id, n.markup)
                     for n in thing.notes
                 ],
+                sources=[
+                    Source(s.citation) for s in thing.sources
+                ],
                 members=[member.concept_id for member in thing.members] if hasattr(thing, 'members') else [],
                 member_of=[member_of.concept_id for member_of in thing.member_of],
                 superordinates=[broader_concept.concept_id for broader_concept in thing.broader_concepts]
@@ -165,6 +172,9 @@ def _from_thing(self, thing):
                     Note(n.note, n.notetype_id, n.language_id, n.markup)
                     for n in thing.notes
                 ],
+                sources=[
+                    Source(s.citation) for s in thing.sources
+                ],
                 broader=[c.concept_id for c in thing.broader_concepts],
                 narrower=[c.concept_id for c in thing.narrower_concepts],
                 related=[c.concept_id for c in thing.related_concepts],

tests/test_models.py

@@ -19,15 +19,18 @@ def _get_target_class(self):
         return Concept
 
     def test_simple(self):
-        from skosprovider_sqlalchemy.models import Label
+        from skosprovider_sqlalchemy.models import Label, Source
         l = Label('Churches', 'prefLabel', 'en')
+        s = Source('Carlisle, P. 2014.')
         c = self._get_target_class()(
             id=1,
             concept_id=1,
-            labels=[l]
+            labels=[l],
+            sources=[s]
         )
-        self.assertEqual(1, c.id)
-        self.assertEqual(l, c.label())
+        assert 1 == c.id
+        assert l == c.label()
+        assert s in c.sources
 
     def test_related(self):
         c1 = self._get_target_class()(
@@ -121,18 +124,21 @@ def _get_target_class(self):
         return ConceptScheme
 
     def test_simple(self):
-        from skosprovider_sqlalchemy.models import Label, Language
+        from skosprovider_sqlalchemy.models import Label, Language, Source
         l = Label('Heritage types', 'prefLabel', 'en')
         en = Language('en', 'English')
+        s = Source('Carlisle, P. 2014.')
         c = self._get_target_class()(
             id=1,
             labels=[l],
-            languages=[en]
+            languages=[en],
+            sources=[s]
         )
-        self.assertEqual(1, c.id)
-        self.assertEqual(l, c.label())
-        self.assertEqual(1, len(c.languages))
-        self.assertIn(en, c.languages)
+        assert 1 == c.id
+        assert l == c.label()
+        assert 1 == len(c.languages)
+        assert en in c.languages
+        assert s in c.sources
 
 
 class CollectionTests(ModelTestCase):
@@ -318,6 +324,30 @@ def test_simple(self):
         self.assertEqual('definition', n.__str__())
 
 
+class TestiSource(DBTestCase):
+
+    def setUp(self):
+        Base.metadata.create_all(self.engine)
+        self.session = self.session_maker()
+        Initialiser(self.session).init_all()
+
+    def tearDown(self):
+        self.session.rollback()
+        self.session.close_all()
+        Base.metadata.drop_all(self.engine)
+
+    def _get_target_class(self):
+        from skosprovider_sqlalchemy.models import Source
+        return Source
+
+    def test_simple(self):
+        s = self._get_target_class()(
+            'Van Daele, K; Meganck M. & Mortier S 2014. Data Driven Systems and System Driven Data.'
+        )
+        assert 'Van Daele, K; Meganck M. & Mortier S 2014. Data Driven Systems and System Driven Data.' == s.citation
+        assert str(s) == s.citation
+
+
 class TestMatchType:
 
     def _get_target_class(self):