Complete DANCER

[achen353]

Context

#20 #21

Summary

1. Fix some bugs in convert_to_extractive.py
   • Generating the test.json and ca_test.json would take some time because batch document processing is not applicable due to the varied number of sections in each document.
2. Fully integrate DANCER for training and evaluation:
   • For training, each (section_text, section_label) becomes a training data point
   • For testing, the model cannot batch process documents because of the varied number of sections of each document. Instead, the model processes one documents at a time: generates a predicted extractive summary for each section, concatenates them, and compares them with the given abstractive summary. Because no back prop involved.

Samples

1. Example entry in train.json (1 document):

{
'src': 
    [
        [  # SECTION 1
            ['Congress', 'makes', 'the', 'following', 'findings', ':', 'It', 'is', 'the', 'policy', 'of', 'the', 'United', 'States', 'to', 'support', 'the', 'right', 'of', 'the', 'people', 'of', 'Iran', 'to', 'seek', ',', 'receive', ',', 'and', 'impart', 'information', 'and', 'ideas', 'through', 'any', 'media', ',', 'regardless', 'of', 'frontiers', ',', 'in', 'accordance', 'with', 'article', '19', 'of', 'the', 'Universal', 'Declaration', 'of', 'Human', 'Rights', '.'], 
            ['Consonant', 'with', 'this', 'policy', ',', 'radio', 'broadcasting', 'to', 'Iran', 'may', 'be', 'effective', 'in', 'furthering', 'the', 'open', 'communication', 'of', 'accurate', 'information', 'and', 'ideas', 'about', 'Iran', 'to', 'the', 'people', 'of', 'Iran', '.'], 
            ['Such', 'broadcasting', 'to', 'Iran', ',', 'operated', 'in', 'a', 'manner', 'not', 'inconsistent', 'with', 'the', 'broad', 'foreign', 'policy', 'of', 'the', 'United', 'States', 'and', 'in', 'accordance', 'with', 'high', 'professional', 'standards', ',', 'would', 'be', 'in', 'the', 'national', 'interest', '.']
        ], 
        [  # SECTION 2
            ['In', 'order', 'to', 'carry', 'out', 'the', 'objectives', 'set', 'forth', 'in', 'section', '2', ',', 'the', 'United', 'States', 'Information', 'Agency', 'shall', 'provide', 'for', 'the', 'open', 'communication', 'of', 'information', 'and', 'ideas', 'on', 'Iran', 'through', 'the', 'use', 'of', 'radio', 'broadcasting', 'to', 'Iran', '.'], 
            ['Radio', 'broadcasting', 'to', 'Iran', 'under', 'this', 'section', 'shall', 'serve', 'as', 'a', 'consistently', 'reliable', 'and', 'authoritative', 'source', 'of', 'accurate', ',', 'objective', ',', 'and', 'comprehensive', 'news', 'on', 'Iran', '.'], 
            ['(', '1', ')', 'Radio', 'broadcasting', 'under', 'subsection', '(', 'a', ')', 'shall', 'be', 'provided', 'in', 'accordance', 'with', 'standards', 'that', 'ensure', 'the', 'broadcast', 'of', 'programs', 'which', 'are', 'objective', ',', 'accurate', ',', 'and', 'balanced', ',', 'and', 'which', 'present', 'a', 'variety', 'of', 'views', '.'], 
            ['Such', 'standards', 'shall', 'be', 'established', 'by', 'the', 'board', 'established', 'under', 'section', '4', '.'], ['Radio', 'broadcasting', 'under', 'subsection', '(', 'a', ')', 'shall', 'be', 'provided', 'in', 'the', 'Farsi', 'language', '.'], ['Any', 'program', 'of', 'United', 'States', 'Government', 'radio', 'broadcasts', 'in', 'the', 'Farsi', 'language', 'under', 'this', 'section', 'shall', 'be', 'designated', '"', 'Radio', 'Free', 'Iran', '"', '.'], 
            ['Relationship', 'With', 'Other', 'Radio', 'Service', 'to', 'Iran', '.'], ['It', 'is', 'the', 'sense', 'of', 'Congress', 'that', 'radio', 'broadcasting', 'under', 'this', 'section', 'supplement', 'and', 'not', 'supplant', 'other', 'radio', 'broadcasting', 'and', 'radio', 'broadcasting', 'services', 'to', 'Iran', 'in', 'the', 'Farsi', 'language', 'that', 'are', 'provided', 'by', 'the', 'United', 'States', 'Government', '.'], 
            ['The', 'Director', 'of', 'the', 'United', 'States', 'Information', 'Agency', 'may', 'carry', 'out', 'this', 'section', 'by', 'means', 'of', 'grants', ',', 'contracts', ',', 'and', 'leases', 'and', 'by', 'such', 'other', 'means', 'as', 'the', 'Director', 'determines', 'appropriate', '.'], 
            ['Any', 'grant', ',', 'contract', ',', 'or', 'lease', 'under', 'this', 'subsection', 'shall', 'specify', 'that', 'payment', 'thereunder', 'by', 'the', 'Director', 'is', 'subject', 'to', 'the', 'availability', 'of', 'appropriations', 'therefor', '.'], ['Assistance', 'From', 'Other', 'Government', 'Agencies', '.'], 
            ['The', 'Director', 'may', 'secure', 'on', 'a', 'reimbursable', 'basis', 'from', 'any', 'department', 'or', 'agency', 'of', 'the', 'Federal', 'Government', ',', 'with', 'the', 'concurrence', 'of', 'the', 'head', 'of', 'the', 'department', 'or', 'agency', ',', 'any', 'technical', 'or', 'administrative', 'support', 'or', 'services', 'that', 'the', 'Director', 'may', 'require', 'in', 'order', 'to', 'provide', 'radio', 'broadcasting', 'to', 'Iran', 'under', 'this', 'section', '.'], 
            ['Any', 'reimbursement', 'under', 'this', 'subsection', 'shall', 'be', 'credited', 'to', 'the', 'appropriation', 'from', 'which', 'the', 'support', 'or', 'services', 'was', 'derived', '.']
        ], 
        [  # SECTION 3
            ['There', 'is', 'hereby', 'established', 'an', 'advisory', 'board', 'to', 'be', 'known', 'as', 'the', 'Advisory', 'Board', 'for', 'Radio', 'Free', 'Iran', '.'], 
            ['(', '1', ')', 'The', 'Board', 'shall', 'be', 'composed', 'of', 'eight', 'members', 'of', 'whom', 'four', 'shall', 'be', 'appointed', 'by', 'the', 'President', '.'], 
            ['Two', 'shall', 'be', 'appointed', 'by', 'the', 'President', 'pro', 'tempore', 'of', 'the', 'Senate', ',', 'upon', 'the', 'recommendation', 'of', 'the', 'majority', 'and', 'minority', 'leaders', 'of', 'the', 'Senate', '.'], 
            ['And', 'two', 'shall', 'be', 'appointed', 'by', 'the', 'Speaker', 'of', 'the', 'House', 'of', 'Representatives', ',', 'in', 'consultation', 'with', 'the', 'minority', 'leader', 'of', 'the', 'House', 'of', 'Representatives', '.'], 
            ['Members', 'shall', 'be', 'appointed', 'for', 'terms', 'of', '4', 'years', '.'], 
            ['Any', 'vacancy', 'in', 'the', 'Board', 'shall', 'not', 'affect', 'its', 'powers', 'but', 'shall', 'be', 'filled', 'in', 'the', 'same', 'manner', 'as', 'the', 'original', 'appointment', '.'], 
            ['The', 'President', 'shall', 'designate', 'one', 'member', 'of', 'the', 'Board', 'to', 'be', 'the', 'Chairman', '.'], ['The', 'Board', 'shall', 'have', 'the', 'following', 'duties', ':', 'To', 'establish', 'standards', 'for', 'the', 'broadcast', 'of', 'programs', 'under', 'section', '3', ',', 'which', 'standards', 'shall', 'ensure', 'that', 'such', 'programs', 'are', 'objective', ',', 'accurate', ',', 'and', 'balanced', ',', 'and', 'present', 'a', 'variety', 'of', 'views', '.'], 
            ['To', 'monitor', 'the', 'broadcast', 'of', 'programs', 'under', 'that', 'section', 'in', 'order', 'to', 'ensure', 'that', 'the', 'programs', 'meet', 'the', 'standards', 'so', 'established', '.'], 
            ['(', '1', ')', 'Each', 'member', 'of', 'the', 'Board', 'who', 'is', 'not', 'an', 'officer', 'or', 'employee', 'of', 'the', 'Federal', 'Government', 'shall', 'be', 'compensated', 'at', 'a', 'rate', 'equal', 'to', 'the', 'daily', 'equivalent', 'of', 'the', 'annual', 'rate', 'of', 'basic', 'pay', 'prescribed', 'for', 'level', 'IV', 'of', 'the', 'Executive', 'Schedule', 'under', 'section', '5315', 'of', 'title', '5', ',', 'United', 'States', 'Code', ',', 'for', 'each', 'day', 'during', 'which', 'such', 'member', 'is', 'engaged', 'in', 'the', 'performance', 'of', 'the', 'duties', 'of', 'the', 'Board', '.'], 
            ['All', 'members', 'of', 'the', 'Board', 'who', 'are', 'officers', 'or', 'employees', 'of', 'the', 'United', 'States', 'shall', 'serve', 'without', 'compensation', 'in', 'addition', 'to', 'that', 'received', 'for', 'their', 'services', 'as', 'officers', 'or', 'employees', 'of', 'the', 'United', 'States', '.'], 
            ['The', 'members', 'of', 'the', 'Board', 'shall', 'be', 'allowed', 'travel', 'expenses', ',', 'including', 'per', 'diem', 'in', 'lieu', 'of', 'subsistence', ',', 'at', 'rates', 'authorized', 'for', 'employees', 'of', 'agencies', 'under', 'subchapter', 'I', 'of', 'chapter', '57', 'of', 'title', '5', ',', 'United', 'States', 'Code', ',', 'while', 'away', 'from', 'their', 'homes', 'or', 'regular', 'places', 'of', 'business', 'in', 'the', 'performance', 'of', 'services', 'for', 'the', 'Board', '.'], 
            ['Procurement', 'of', 'Supplies', 'and', 'Services', '.'], 
            ['The', 'Board', 'may', ',', 'to', 'the', 'extent', 'it', 'considers', 'necessary', 'to', 'carry', 'out', 'its', 'duties', 'under', 'this', 'section', ',', 'procure', 'supplies', ',', 'services', ',', 'and', 'other', 'personal', 'property', ',', 'including', 'specialized', 'electronic', 'equipment', '.']
        ]
    ], 
'labels': 
    [
        [0, 1, 0],  # SECTION 1
        [1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],  # SECTION 2
        [1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0]  # SECTION 3
    ]
}

2. Corresponding entry in train.txt (1 section in vector):
   HOWEVER, for training and validation data, when converting from json to txt, we treat each section instead of a document as a single data point. So this example is just SECTION 1 of the document presented by the json above.

{
"input_ids": 
    [101, 3519, 3084, 1996, 2206, 9556, 1024, 2009, 2003, 1996, 3343, 1997, 1996, 2142, 2163, 2000, 2490, 1996, 2157, 1997, 1996, 2111, 1997, 4238, 2000, 6148, 1010, 4374, 1010, 1998, 17727, 8445, 2592, 1998, 4784, 2083, 2151, 2865, 1010, 7539, 1997, 28750, 1010, 1999, 10388, 2007, 3720, 2539, 1997, 1996, 5415, 8170, 1997, 2529, 2916, 1012, 102, 101, 18265, 2007, 2023, 3343, 1010, 2557, 5062, 2000, 4238, 2089, 2022, 4621, 1999, 2582, 2075, 1996, 2330, 4807, 1997, 8321, 2592, 1998, 4784, 2055, 4238, 2000, 1996, 2111, 1997, 4238, 1012, 102, 101, 2107, 5062, 2000, 4238, 1010, 3498, 1999, 1037, 5450, 2025, 20316, 2007, 1996, 5041, 3097, 3343, 1997, 1996, 2142, 2163, 1998, 1999, 10388, 2007, 2152, 2658, 4781, 1010, 2052, 2022, 1999, 1996, 2120, 3037, 1012, 102], 
"token_type_ids": 
    [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 
"labels": 
    [0, 1, 0], 
"sent_rep_token_ids": 
    [0, 57, 90], 
"sent_lengths": 
    [57, 33, 37]
}

3. Example entry in ca_test.json (1 document):

{
'src':
    [
        [  # SECTION 1
            ['The', 'Legislature', 'finds', 'and', 'declares', 'all', 'of', 'the', 'following', ':', '(', '1', ')', 'Since', '1899', 'congressionally', 'chartered', 'veterans', '’', 'organizations', 'have', 'provided', 'a', 'valuable', 'service', 'to', 'our', 'nation', '’s', 'returning', 'service', 'members', '.'], 
            ['These', 'organizations', 'help', 'preserve', 'the', 'memories', 'and', 'incidents', 'of', 'the', 'great', 'hostilities', 'fought', 'by', 'our', 'nation', ',', 'and', 'preserve', 'and', 'strengthen', 'comradeship', 'among', 'members', '.'], 
            ['These', 'veterans', '’', 'organizations', 'also', 'own', 'and', 'manage', 'various', 'properties', 'including', 'lodges', ',', 'posts', ',', 'and', 'fraternal', 'halls', '.'], 
            ['These', 'properties', 'act', 'as', 'a', 'safe', 'haven', 'where', 'veterans', 'of', 'all', 'ages', 'and', 'their', 'families', 'can', 'gather', 'together', 'to', 'find', 'camaraderie', 'and', 'fellowship', ',', 'share', 'stories', ',', 'and', 'seek', 'support', 'from', 'people', 'who', 'understand', 'their', 'unique', 'experiences', '.'], 
            ['This', 'aids', 'in', 'the', 'healing', 'process', 'for', 'these', 'returning', 'veterans', ',', 'and', 'ensures', 'their', 'health', 'and', 'happiness', '.'], 
            ['As', 'a', 'result', 'of', 'congressional', 'chartering', 'of', 'these', 'veterans', '’', 'organizations', ',', 'the', 'United', 'States', 'Internal', 'Revenue', 'Service', 'created', 'a', 'special', 'tax', 'exemption', 'for', 'these', 'organizations', 'under', 'Section', '501(c)(19', ')', 'of', 'the', 'Internal', 'Revenue', 'Code', '.'], 
            ['Section', '501(c)(19', ')', 'of', 'the', 'Internal', 'Revenue', 'Code', 'and', 'related', 'federal', 'regulations', 'provide', 'for', 'the', 'exemption', 'for', 'posts', 'or', 'organizations', 'of', 'war', 'veterans', ',', 'or', 'an', 'auxiliary', 'unit', 'or', 'society', 'of', ',', 'or', 'a', 'trust', 'or', 'foundation', 'for', ',', 'any', 'such', 'post', 'or', 'organization', 'that', ',', 'among', 'other', 'attributes', ',', 'carries', 'on', 'programs', 'to', 'perpetuate', 'the', 'memory', 'of', 'deceased', 'veterans', 'and', 'members', 'of', 'the', 'Armed', 'Forces', 'and', 'to', 'comfort', 'their', 'survivors', ',', 'conducts', 'programs', 'for', 'religious', ',', 'charitable', ',', 'scientific', ',', 'literary', ',', 'or', 'educational', 'purposes', ',', 'sponsors', 'or', 'participates', 'in', 'activities', 'of', 'a', 'patriotic', 'nature', ',', 'and', 'provides', 'social', 'and', 'recreational', 'activities', 'for', 'their', 'members', '.'], 
            ['Section', '215.1', 'of', 'the', 'Revenue', 'and', 'Taxation', 'Code', 'stipulates', 'that', 'all', 'buildings', ',', 'support', 'and', 'so', 'much', 'of', 'the', 'real', 'property', 'on', 'which', 'the', 'buildings', 'are', 'situated', 'as', 'may', 'be', 'required', 'for', 'the', 'convenient', 'use', 'and', 'occupation', 'of', 'the', 'buildings', ',', 'used', 'exclusively', 'for', 'charitable', 'purposes', ',', 'owned', 'by', 'a', 'veterans', '’', 'organization', 'that', 'has', 'been', 'chartered', 'by', 'the', 'Congress', 'of', 'the', 'United', 'States', ',', 'organized', 'and', 'operated', 'for', 'charitable', 'purposes', ',', 'when', 'the', 'same', 'are', 'used', 'solely', 'and', 'exclusively', 'for', 'the', 'purpose', 'of', 'the', 'organization', ',', 'if', 'not', 'conducted', 'for', 'profit', 'and', 'no', 'part', 'of', 'the', 'net', 'earnings', 'of', 'which', 'ensures', 'to', 'the', 'benefit', 'of', 'any', 'private', 'individual', 'or', 'member', 'thereof', ',', 'are', 'exempt', 'from', 'taxation', '.'], 
            ['The', 'Chief', 'Counsel', 'of', 'the', 'State', 'Board', 'of', 'Equalization', 'concluded', ',', 'based', 'on', 'a', '1979', 'appellate', 'court', 'decision', ',', 'that', 'only', 'parts', 'of', 'American', 'Legion', 'halls', 'are', 'exempt', 'from', 'property', 'taxation', 'and', 'that', 'other', 'parts', ',', 'such', 'as', 'billiard', 'rooms', ',', 'card', 'rooms', ',', 'and', 'similar', 'areas', ',', 'are', 'not', 'exempt', '.'], 
            ['In', 'a', '1994', 'memorandum', ',', 'the', 'State', 'Board', 'of', 'Equalization', '’s', 'legal', 'division', 'further', 'concluded', 'that', 'the', 'areas', 'normally', 'considered', 'eligible', 'for', 'exemptions', 'are', 'the', 'office', 'areas', 'used', 'to', 'counsel', 'veterans', 'and', 'the', 'area', 'used', 'to', 'store', 'veterans', '’', 'records', ',', 'but', 'that', 'the', 'meeting', 'hall', 'and', 'bar', 'found', 'in', 'most', 'of', 'the', 'facilities', 'are', 'not', 'considered', 'used', 'for', 'charitable', 'purposes', '.'], 
            ['Tax', '-', 'exempt', 'status', 'is', 'intended', 'to', 'provide', 'economic', 'incentive', 'and', 'support', 'to', 'veterans', '’', 'organizations', 'to', 'provide', 'for', 'the', 'social', 'welfare', 'of', 'the', 'community', 'of', 'current', 'and', 'former', 'military', 'personnel', '.'], 
            ['The', 'State', 'Board', 'of', 'Equalization', '’s', 'constriction', 'of', 'the', 'tax', 'exemption', 'has', 'resulted', 'in', 'an', 'onerous', 'tax', 'burden', 'on', 'California', 'veteran', 'service', 'organizations', 'posts', 'or', 'halls', ',', 'hinders', 'the', 'posts', '’', 'ability', 'to', 'provide', 'facilities', 'for', 'veterans', ',', 'and', 'threatens', 'the', 'economic', 'viability', 'of', 'many', 'local', 'organizations', '.'], 
            ['The', 'charitable', 'activities', 'of', 'a', 'veteran', 'service', 'organizations', 'post', 'or', 'hall', 'are', 'much', 'more', 'than', 'the', 'counseling', 'of', 'veterans', '.'], 
            ['The', 'requirements', 'listed', 'for', 'qualification', 'for', 'the', 'federal', 'tax', 'exemption', 'clearly', 'dictate', 'a', 'need', 'for', 'more', 'than', 'just', 'an', 'office', '.'], 
            ['Programs', 'to', 'perpetuate', 'the', 'memory', 'of', 'deceased', 'veterans', 'and', 'members', 'of', 'the', 'Armed', 'Forces', 'and', 'to', 'comfort', 'their', 'survivors', 'require', 'the', 'use', 'of', 'facilities', 'for', 'funerals', 'and', 'receptions', '.'], ['Programs', 'for', 'religious', ',', 'charitable', ',', 'scientific', ',', 'literary', ',', 'or', 'educational', 'purposes', 'require', 'space', 'for', 'more', 'than', '50', 'attendees', '.'], 
            ['Activities', 'of', 'a', 'patriotic', 'nature', 'need', 'facilities', 'to', 'accommodate', 'hundreds', 'of', 'people', '.'], ['Social', 'and', 'recreational', 'activities', 'for', 'members', 'require', 'precisely', 'those', 'areas', 'considered', '“', 'not', 'used', 'for', 'charitable', 'purposes', '”', 'by', 'the', 'State', 'Board', 'of', 'Equalization', '.'], 
            ['The', 'State', 'Board', 'of', 'Equalization', '’s', 'interpretation', 'of', 'the', 'Revenue', 'and', 'Taxation', 'Code', 'reflects', 'a', 'lack', 'of', 'understanding', 'of', 'the', 'purpose', 'and', 'programs', 'of', 'the', 'veterans', 'service', 'organizations', 'posts', 'or', 'halls', 'and', 'is', 'detrimental', 'to', 'the', 'good', 'works', 'performed', 'in', 'support', 'of', 'our', 'veteran', 'community', '.']
        ], 
        [  # SECTION 2
            ['Section', '215.1', 'of', 'the', 'Revenue', 'and', 'Taxation', 'Code', 'is', 'amended', 'to', 'read', ':', '215.1', '.'], ['All', 'buildings', ',', 'and', 'so', 'much', 'of', 'the', 'real', 'property', 'on', 'which', 'the', 'buildings', 'are', 'situated', 'as', 'may', 'be', 'required', 'for', 'the', 'convenient', 'use', 'and', 'occupation', 'of', 'the', 'buildings', ',', 'used', 'exclusively', 'for', 'charitable', 'purposes', ',', 'owned', 'by', 'a', 'veterans', '’', 'organization', 'that', 'has', 'been', 'chartered', 'by', 'the', 'Congress', 'of', 'the', 'United', 'States', ',', 'organized', 'and', 'operated', 'for', 'charitable', 'purposes', ',', 'and', 'exempt', 'from', 'federal', 'income', 'tax', 'as', 'an', 'organization', 'described', 'in', 'Section', '501(c)(19', ')', 'of', 'the', 'Internal', 'Revenue', 'Code', 'when', 'the', 'same', 'are', 'used', 'solely', 'and', 'exclusively', 'for', 'the', 'purpose', 'of', 'the', 'organization', ',', 'if', 'not', 'conducted', 'for', 'profit', 'and', 'no', 'part', 'of', 'the', 'net', 'earnings', 'of', 'which', 'inures', 'to', 'the', 'benefit', 'of', 'any', 'private', 'individual', 'or', 'member', 'thereof', ',', 'shall', 'be', 'exempt', 'from', 'taxation', '.'], 
            ['The', 'exemption', 'provided', 'for', 'in', 'this', 'section', 'shall', 'apply', 'to', 'the', 'property', 'of', 'all', 'organizations', 'meeting', 'the', 'requirements', 'of', 'this', 'section', ',', 'subdivision', '(', 'b', ')', 'of', 'Section', '4', 'of', 'Article', 'XIII', 'of', 'the', 'California', 'Constitution', ',', 'and', 'paragraphs', '(', '1', ')', 'to', '(', '4', ')', ',', 'inclusive', ',', '(', '6', ')', ',', 'and', '(', '7', ')', 'of', 'subdivision', '(', 'a', ')', 'of', 'Section', '214', '.'], 
            ['The', 'exemption', 'specified', 'by', 'subdivision', '(', 'a', ')', 'shall', 'not', 'be', 'denied', 'to', 'a', 'property', 'on', 'the', 'basis', 'that', 'the', 'property', 'is', 'used', 'for', 'fraternal', ',', 'lodge', ',', 'or', 'social', 'club', 'purposes', '.'], 
            ['With', 'regard', 'to', 'this', 'subdivision', ',', 'the', 'Legislature', 'finds', 'and', 'declares', 'all', 'of', 'the', 'following', ':', 'The', 'exempt', 'activities', 'of', 'a', 'veterans', '’', 'organization', 'as', 'described', 'in', 'subdivision', '(', 'a', ')', 'qualitatively', 'differ', 'from', 'the', 'exempt', 'activities', 'of', 'other', 'nonprofit', 'entities', 'that', 'use', 'property', 'for', 'fraternal', ',', 'lodge', ',', 'or', 'social', 'club', 'purposes', 'in', 'that', 'the', 'exempt', 'purpose', 'of', 'the', 'veterans', '’', 'organization', 'is', 'to', 'conduct', 'programs', 'to', 'perpetuate', 'the', 'memory', 'of', 'deceased', 'veterans', 'and', 'members', 'of', 'the', 'Armed', 'Forces', 'and', 'to', 'comfort', 'their', 'survivors', ',', 'to', 'conduct', 'programs', 'for', 'religious', ',', 'charitable', ',', 'scientific', ',', 'literary', ',', 'or', 'educational', 'purposes', ',', 'to', 'sponsor', 'or', 'participate', 'in', 'activities', 'of', 'a', 'patriotic', 'nature', ',', 'and', 'to', 'provide', 'social', 'and', 'recreational', 'activities', 'for', 'their', 'members', '.'], 
            ['In', 'light', 'of', 'this', 'distinction', ',', 'the', 'use', 'of', 'real', 'property', 'by', 'a', 'veterans', '’', 'organization', 'as', 'described', 'in', 'subdivision', '(', 'a', ')', ',', 'for', 'fraternal', ',', 'lodge', ',', 'or', 'social', 'club', 'purposes', 'is', 'central', 'to', 'that', 'organization', '’s', 'exempt', 'purposes', 'and', 'activities', '.'], 
            ['In', 'light', 'of', 'the', 'factors', 'set', 'forth', 'in', 'subparagraphs', '(', 'A', ')', 'and', '(', 'B', ')', ',', 'the', 'use', 'of', 'real', 'property', 'by', 'a', 'veterans', '’', 'organization', 'as', 'described', 'in', 'subdivision', '(', 'a', ')', 'for', 'fraternal', ',', 'lodge', ',', 'or', 'social', 'club', 'purposes', ',', 'constitutes', 'the', 'exclusive', 'use', 'of', 'that', 'property', 'for', 'a', 'charitable', 'purpose', 'within', 'the', 'meaning', 'of', 'subdivision', '(', 'b', ')', 'of', 'Section', '4', 'of', 'Article', 'XIII', 'of', 'the', 'California', 'Constitution', '.'], 
            ['The', 'exemption', 'provided', 'for', 'in', 'this', 'section', 'shall', 'not', 'apply', 'to', 'any', 'portion', 'of', 'a', 'property', 'that', 'consists', 'of', 'a', 'bar', 'where', 'alcoholic', 'beverages', 'are', 'served', '.'], 
            ['The', 'portion', 'of', 'the', 'property', 'ineligible', 'for', 'the', 'veterans', '’', 'organization', 'exemption', 'shall', 'be', 'that', 'area', 'used', 'primarily', 'to', 'prepare', 'and', 'serve', 'alcoholic', 'beverages', '.'], 
            ['An', 'organization', 'that', 'files', 'a', 'claim', 'for', 'the', 'exemption', 'provided', 'for', 'in', 'this', 'section', 'shall', 'file', 'with', 'the', 'assessor', 'a', 'valid', 'organizational', 'clearance', 'certificate', 'issued', 'pursuant', 'to', 'Section', '254.6', '.'], 
            ['This', 'exemption', 'shall', 'be', 'known', 'as', 'the', '“', 'veterans', '’', 'organization', 'exemption', '.', '”']
        ]
    ], 
'labels': 
    [
        [0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],  # SECTION 1
        [1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0]  # SECTION 2
    ], 
'tgt':  # One single string (abstractive summary) for the whole document
    'Existing property tax law establishes a veterans ’ organization exemption under which property is exempt from taxation if , among other things , that property is used exclusively for charitable purposes and is owned by a veterans ’ organization .<q>This bill would provide that the veterans ’ organization exemption shall not be denied to a property on the basis that the property is used for fraternal , lodge , or social club purposes , and would make specific findings and declarations in that regard .<q>The bill would also provide that the exemption shall not apply to any portion of a property that consists of a bar where alcoholic beverages are served .<q>Section 2229 of the Revenue and Taxation Code requires the Legislature to reimburse local agencies annually for certain property tax revenues lost as a result of any exemption or classification of property for purposes of ad valorem property taxation .<q>This bill would provide that , notwithstanding Section 2229 of the Revenue and Taxation Code , no appropriation is made and the state shall not reimburse local agencies for property tax revenues lost by them pursuant to the bill .<q>This bill would take effect immediately as a tax levy .'
}

4. Corresponding entry in ca_test.txt (same data as above, 1 document, but converted to vectors):
   Because this is for testing, each document remain intact (instead of each section becomes a training data point)

{
"input_ids": 
    [
        # SECTION 1
        [101, 2930, 6036, 12521, 1012, 4293, 1997, 1996, 2231, 3642, 2003, 13266, 2000, 3191, 1024, 6036, 12521, 1012, 4293, 1012, 102, 101, 2045, 2003, 2182, 3762, 2580, 1999, 1996, 2110, 9837, 1037, 2569, 4636, 2170, 1996, 1523, 2796, 10355, 6599, 6631, 3404, 4636, 1524, 2005, 1996, 24306, 1998, 12816, 1997, 2769, 2015, 2363, 2011, 1996, 2110, 2013, 2796, 6946, 27081, 2000, 1996, 3408, 1997, 8807, 1011, 2110, 10355, 9233, 2015, 2005, 1996, 3800, 1997, 2437, 20611, 2000, 7792, 7799, 2796, 6946, 1012, 102, 101, 2769, 2015, 1999, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 4618, 2022, 2800, 2000, 1996, 2662, 12219, 2491, 3222, 1010, 2588, 10439, 26121, 2011, 1996, 6372, 1010, 2005, 1996, 3800, 1997, 2437, 20611, 2000, 7792, 7799, 2796, 6946, 1010, 1999, 10388, 2007, 4353, 3488, 9675, 1999, 8807, 1011, 2110, 10355, 9233, 2015, 1012, 102], 
        # SECTION 2
        [101, 2930, 6036, 12521, 1012, 3938, 1997, 1996, 2231, 3642, 2003, 13266, 2000, 3191, 1024, 6036, 12521, 1012, 3938, 1012, 102, 101, 2005, 2169, 10807, 2095, 25819, 2007, 1996, 2355, 1516, 2459, 10807, 2095, 1010, 2035, 1997, 1996, 2206, 4618, 6611, 1024, 2006, 2030, 2077, 1996, 2154, 1997, 1996, 2089, 5166, 13921, 2005, 2169, 10807, 2095, 1010, 1996, 2662, 12219, 2491, 3222, 4618, 5646, 1996, 11436, 2561, 3815, 1997, 2460, 28067, 1999, 7909, 3497, 2000, 5258, 1999, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 2005, 1996, 2279, 10807, 2095, 1010, 1998, 4618, 3073, 2000, 1996, 2837, 1999, 1996, 4001, 1998, 3320, 2008, 10592, 1996, 2110, 5166, 2019, 10197, 1997, 1996, 3815, 2734, 2000, 4651, 2013, 1996, 2796, 10355, 2569, 4353, 4636, 2000, 2067, 8873, 3363, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 2005, 1996, 2279, 10807, 2095, 1012, 102, 101, 1996, 11436, 2561, 3815, 1997, 2460, 28067, 2000, 2022, 4015, 2013, 1996, 2796, 10355, 2569, 4353, 4636, 2000, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 4618, 2022, 4340, 2011, 1996, 2662, 12219, 2491, 3222, 2004, 4076, 1024, 102, 101, 1996, 11436, 2193, 1997, 7792, 7799, 2796, 6946, 2008, 2097, 2022, 7792, 2000, 4374, 10504, 2005, 1996, 2279, 10807, 2095, 1010, 28608, 2011, 2028, 2454, 2028, 3634, 4595, 6363, 1006, 1002, 1015, 1010, 2531, 1010, 2199, 1007, 1010, 2007, 2008, 4031, 4359, 2011, 1996, 3815, 11436, 2000, 2022, 3825, 2011, 1996, 6946, 3495, 2046, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 2005, 1996, 2279, 10807, 2095, 1012, 102, 101, 2005, 5682, 1997, 2023, 2930, 1998, 2930, 6036, 12521, 1012, 4293, 1010, 1523, 7792, 7799, 2796, 5917, 1524, 2965, 1037, 2512, 9006, 19498, 2102, 1010, 2512, 26517, 3070, 1010, 2030, 3132, 1011, 10355, 5917, 1010, 2004, 4225, 1999, 1996, 8807, 1011, 2110, 10355, 9233, 2015, 17673, 1998, 1999, 3466, 2004, 3024, 1999, 12572, 1006, 1042, 1007, 1997, 2930, 2539, 1997, 3720, 4921, 1997, 1996, 2662, 4552, 1012, 102, 101, 2023, 3815, 4618, 2022, 2241, 2588, 5025, 10504, 2363, 2046, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 1996, 3025, 10807, 2095, 1010, 2007, 24081, 2081, 2349, 2000, 16051, 2000, 4493, 8807, 1011, 2110, 10355, 9233, 2015, 2030, 4397, 6472, 8807, 1011, 2110, 10355, 9233, 2015, 2007, 4847, 2000, 10504, 2000, 2022, 2081, 2000, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 1012, 102, 101, 1996, 6372, 4618, 4651, 2013, 1996, 2796, 10355, 2569, 4353, 4636, 2000, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 2019, 3815, 7182, 2005, 2169, 7792, 7799, 2796, 5917, 2000, 4374, 1037, 2561, 2025, 2000, 13467, 2048, 3634, 10920, 1011, 2274, 4595, 6363, 1006, 1002, 17528, 1010, 2199, 1007, 2005, 2169, 4284, 1999, 1996, 2279, 10807, 2095, 2008, 2019, 7792, 7799, 2796, 5917, 2003, 7792, 2000, 4374, 2769, 2015, 1010, 2005, 1037, 2561, 2025, 2000, 13467, 2028, 2454, 2028, 3634, 4595, 6363, 1006, 1002, 1015, 1010, 2531, 1010, 2199, 1007, 2005, 1996, 2972, 10807, 2095, 1012, 102, 101, 1996, 2662, 12219, 2491, 3222, 4618, 2191, 12174, 10504, 2013, 1996, 2796, 10355, 6599, 6631, 3404, 4636, 2000, 2169, 7792, 7799, 2796, 5917, 2306, 3429, 2420, 1997, 1996, 2203, 1997, 2169, 10807, 4284, 1012, 102]
    ], 
"token_type_ids": 
    [
        # SECTION 1
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 
        # SECTION 2
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    ], 
"labels": 
    [
        [0, 1, 1],  # SECTION 1
        [0, 1, 0, 0, 1, 0, 0, 0]  # SECTION 2
    ], 
"sent_rep_token_ids": 
    [
        [0, 21, 83],  # SECTION 1
        [0, 21, 139, 178, 247, 316, 379, 476]  # SECTION 2
    ], 
"sent_lengths": 
    [
        [21, 62, 55],  # SECTION 1
        [21, 118, 39, 69, 69, 63, 97, 36]  # SECTION 2
    ], 
"source": 
    [
        # SECTION 1
        ["Section 12012.75 of the Government Code is amended to read : 12012.75 .", "There is hereby created in the State Treasury a special fund called the \\u201c Indian Gaming Revenue Sharing Trust Fund \\u201d for the receipt and deposit of moneys received by the state from Indian tribes pursuant to the terms of tribal - state gaming compacts for the purpose of making distributions to eligible recipient Indian tribes .", "Moneys in the Indian Gaming Revenue Sharing Trust Fund shall be available to the California Gambling Control Commission , upon appropriation by the Legislature , for the purpose of making distributions to eligible recipient Indian tribes , in accordance with distribution plans specified in tribal - state gaming compacts ."], 
        # SECTION 2
        ["Section 12012.90 of the Government Code is amended to read : 12012.90 .", "For each fiscal year commencing with the 2016\\u201317 fiscal year , all of the following shall apply : On or before the day of the May budget revision for each fiscal year , the California Gambling Control Commission shall determine the anticipated total amount of shortfalls in payment likely to occur in the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year , and shall provide to the committee in the Senate and Assembly that considers the State Budget an estimate of the amount needed to transfer from the Indian Gaming Special Distribution Fund to backfill the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year .", "The anticipated total amount of shortfalls to be transferred from the Indian Gaming Special Distribution Fund to the Indian Gaming Revenue Sharing Trust Fund shall be determined by the California Gambling Control Commission as follows :", "The anticipated number of eligible recipient Indian tribes that will be eligible to receive payments for the next fiscal year , multiplied by one million one hundred thousand dollars ( $ 1,100,000 ) , with that product reduced by the amount anticipated to be paid by the tribes directly into the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year .", "For purposes of this section and Section 12012.75 , \\u201c eligible recipient Indian tribe \\u201d means a noncompact , nongaming , or limited - gaming tribe , as defined in the tribal - state gaming compacts ratified and in effect as provided in subdivision ( f ) of Section 19 of Article IV of the California Constitution .", "This amount shall be based upon actual payments received into the Indian Gaming Revenue Sharing Trust Fund the previous fiscal year , with adjustments made due to amendments to existing tribal - state gaming compacts or newly executed tribal - state gaming compacts with respect to payments to be made to the Indian Gaming Revenue Sharing Trust Fund .", "The Legislature shall transfer from the Indian Gaming Special Distribution Fund to the Indian Gaming Revenue Sharing Trust Fund an amount sufficient for each eligible recipient Indian tribe to receive a total not to exceed two hundred seventy - five thousand dollars ( $ 275,000 ) for each quarter in the next fiscal year that an eligible recipient Indian tribe is eligible to receive moneys , for a total not to exceed one million one hundred thousand dollars ( $ 1,100,000 ) for the entire fiscal year .", "The California Gambling Control Commission shall make quarterly payments from the Indian Gaming Revenue Sharing Trust Fund to each eligible recipient Indian tribe within 45 days of the end of each fiscal quarter .", "If the transfer of funds from the Indian Gaming Special Distribution Fund to the Indian Gaming Revenue Sharing Trust Fund results in a surplus , the funds shall remain in the Indian Gaming Revenue Sharing Trust Fund for disbursement in future years , and if necessary , adjustments shall be made to future distributions from the Indian Gaming Special Distribution Fund to the Revenue Sharing Trust Fund .", "In the event the amount appropriated for the fiscal year is insufficient to ensure each eligible recipient Indian tribe receives the total of two hundred seventy - five thousand dollars ( $ 275,000 ) for each fiscal quarter , the Department of Finance , after consultation with the California Gambling Control Commission , shall submit to the Legislature a request for a budget augmentation for the current fiscal year with an explanation as to the reason why the amount appropriated for the fiscal year was insufficient .", "At the end of each fiscal quarter , the California Gambling Control Commission \\u2019s Indian Gaming Revenue Sharing Trust Fund report shall include information that identifies each of the eligible recipient Indian tribes for that fiscal quarter , the amount paid into the Indian Gaming Revenue Sharing Trust Fund by each of the tribes pursuant to the applicable sections of the tribal - state gaming compact , provided that tribes contributing on a net win or gross gaming revenue basis may be aggregated in the report , and the amount necessary to backfill from the Indian Gaming Special Distribution Fund the shortfall in the Indian Gaming Revenue Sharing Trust Fund in order for each eligible recipient Indian tribe to receive the total of two hundred seventy - five thousand dollars ( $ 275,000 ) for the fiscal quarter ."]
    ], 
"target": 
    [
        # SECTION 1
        "Existing federal law , the Indian Gaming Regulatory Act of 1988 , provides for the negotiation and execution of tribal - state gaming compacts for the purpose of authorizing certain types of gaming on Indian lands within a state .<q>The California Constitution authorizes the Governor to negotiate and conclude compacts , subject to ratification by the Legislature .<q>Existing law expressly ratifies a number of tribal - state gaming compacts , and amendments of tribal - state gaming compacts , between the State of California and specified Indian tribes .<q>Existing law establishes the Indian Gaming Revenue Sharing Trust Fund within the State Treasury for the receipt and deposit of moneys derived from gaming device license fees that are received from tribes pursuant to the terms of tribal - state gaming compacts for the purpose of making distributions to noncompact tribes .<q>Existing law provides that moneys in that fund are available to the California Gambling Control Commission , upon appropriation by the Legislature , for the purpose of making those distributions in accordance with plans specified in tribal - state gaming compacts .<q>This bill would clarify that the purpose of the fund is for making distributions to eligible recipient Indian tribes .<q>Existing law requires the California Gambling Control Commission to , on or before the day of the May budget revision for each fiscal year , determine the anticipated total amount of shortfalls in payment likely to occur in the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year , and to provide to the committee in the Senate and Assembly that considers the State Budget an estimate of the amount needed to transfer from the Indian Gaming Special Distribution Fund to backfill the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year .<q>Existing law requires , at the end of each fiscal quarter , the commission \\u2019s Indian Gaming Revenue Sharing Trust Fund report to include specified information , including , among other things , the amount paid into the Indian Gaming Revenue Sharing Trust Fund by each of the tribes pursuant to the applicable sections of the tribal - state gaming compact .<q>This bill would provide that tribes contributing to the Indian Gaming Revenue Sharing Trust Fund on a net win or gross gaming revenue basis may be aggregated in the quarterly report described above .<q>Existing law requires the California Gambling Control Commission to determine the amount of money needed to be transferred from the Indian Gaming Special Distribution Fund to the Indian Gaming Revenue Sharing Trust Fund to ensure that each eligible recipient Indian tribe receives a specified amount of the funds .<q>Existing law defines \\u201c eligible recipient tribe \\u201d for those purposes to mean a noncompact tribe , as defined in the tribal - state gaming compacts ratified and in effect , as specified .<q>Those compacts define \\u201c noncompact tribe \\u201d to mean a federally recognized tribe that operates fewer than 350 gaming devices .<q>This bill would clarify that \\u201c eligible recipient Indian tribe \\u201d means a noncompact , nongaming , or limited - gaming tribe , as defined in the tribal - state gaming compacts ratified and in effect , as provided .<q>The bill would delete other related , obsolete provisions .", 
        # SECTION 2
        "Existing federal law , the Indian Gaming Regulatory Act of 1988 , provides for the negotiation and execution of tribal - state gaming compacts for the purpose of authorizing certain types of gaming on Indian lands within a state .<q>The California Constitution authorizes the Governor to negotiate and conclude compacts , subject to ratification by the Legislature .<q>Existing law expressly ratifies a number of tribal - state gaming compacts , and amendments of tribal - state gaming compacts , between the State of California and specified Indian tribes .<q>Existing law establishes the Indian Gaming Revenue Sharing Trust Fund within the State Treasury for the receipt and deposit of moneys derived from gaming device license fees that are received from tribes pursuant to the terms of tribal - state gaming compacts for the purpose of making distributions to noncompact tribes .<q>Existing law provides that moneys in that fund are available to the California Gambling Control Commission , upon appropriation by the Legislature , for the purpose of making those distributions in accordance with plans specified in tribal - state gaming compacts .<q>This bill would clarify that the purpose of the fund is for making distributions to eligible recipient Indian tribes .<q>Existing law requires the California Gambling Control Commission to , on or before the day of the May budget revision for each fiscal year , determine the anticipated total amount of shortfalls in payment likely to occur in the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year , and to provide to the committee in the Senate and Assembly that considers the State Budget an estimate of the amount needed to transfer from the Indian Gaming Special Distribution Fund to backfill the Indian Gaming Revenue Sharing Trust Fund for the next fiscal year .<q>Existing law requires , at the end of each fiscal quarter , the commission \\u2019s Indian Gaming Revenue Sharing Trust Fund report to include specified information , including , among other things , the amount paid into the Indian Gaming Revenue Sharing Trust Fund by each of the tribes pursuant to the applicable sections of the tribal - state gaming compact .<q>This bill would provide that tribes contributing to the Indian Gaming Revenue Sharing Trust Fund on a net win or gross gaming revenue basis may be aggregated in the quarterly report described above .<q>Existing law requires the California Gambling Control Commission to determine the amount of money needed to be transferred from the Indian Gaming Special Distribution Fund to the Indian Gaming Revenue Sharing Trust Fund to ensure that each eligible recipient Indian tribe receives a specified amount of the funds .<q>Existing law defines \\u201c eligible recipient tribe \\u201d for those purposes to mean a noncompact tribe , as defined in the tribal - state gaming compacts ratified and in effect , as specified .<q>Those compacts define \\u201c noncompact tribe \\u201d to mean a federally recognized tribe that operates fewer than 350 gaming devices .<q>This bill would clarify that \\u201c eligible recipient Indian tribe \\u201d means a noncompact , nongaming , or limited - gaming tribe , as defined in the tribal - state gaming compacts ratified and in effect , as provided .<q>The bill would delete other related , obsolete provisions ."
    ]
}

[achen353]

Here's the processed data:
https://drive.google.com/file/d/1GaQT0bMggJ_vR1njdmnvA8pWgiV5gtq0/view?usp=sharing
@stephanieeechang @andywang268