Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for bulk creation of objects via API #1553

Closed
dirtycajunrice opened this issue Oct 2, 2017 · 5 comments
Closed

Support for bulk creation of objects via API #1553

dirtycajunrice opened this issue Oct 2, 2017 · 5 comments
Labels
type: feature Introduction of new functionality to the application
Milestone

Comments

@dirtycajunrice
Copy link
Contributor

dirtycajunrice commented Oct 2, 2017

Issue type

[X] Feature request
[ ] Bug report
[ ] Documentation

Description

Multi-entry API calls would make automation much easier. I have scripted a "New Customer.py" which creates the Tenant, VRF, 10xVLANs, 10xPrefixes, and about 50 initial IPs. Currently, my options are to paste it out to CSV and import it section by section or to create a loop for the HTTP PUSH request which is a nightmare for error checking.

Browsable API example:

[
    {
        "name": "Cust 2",
        "slug": "cust-2",
        "group": 1
    },
    {
        "name": "Cust 3",
        "slug": "cust-3",
        "group": 1
    }
]

Thoughts?

@Zimeon-
Copy link

Zimeon- commented Oct 3, 2017

Ran into the same issue that it didn't accept multiple entries when posting a JSON. Your example is missing [] and a comma between the entries but none the less it doesn't support it. Ended up just making multiple post entries in the script but bulk data in a single json would be nice.

@dirtycajunrice
Copy link
Contributor Author

@Zimeon- fixed example. Thanks! Do you have request.error_code and request.response data from each push? And if so how are you logically checking those? Dict?

@Zimeon-
Copy link

Zimeon- commented Oct 4, 2017

I simply stored the response code and moved along. After that I manually verified the failed ones by hand as I just had to import stuff once from an old db to netbox.

@jeremystretch
Copy link
Member

This seems simple enough to implement in the writable serializers. We might want to restrict the total number of objects able to be created in one request though: It might be reasonable to reuse the MAX_PAGE_SIZE configuration parameter for that purpose.

@jeremystretch jeremystretch added API change type: feature Introduction of new functionality to the application labels Oct 16, 2017
@dirtycajunrice
Copy link
Contributor Author

Yes i agree that MAX_PAGE_SIZE would safely limit to (500?) by default

@jeremystretch jeremystretch added this to the v2.3 milestone Nov 7, 2017
@jeremystretch jeremystretch changed the title Support for bulk API import Support for bulk creation of objects via API Nov 7, 2017
funzoneq added a commit to openfibernet/netbox that referenced this issue Mar 8, 2018
* Closes netbox-community#1553: Introduced support for bulk object creation via the API

* Removed prefix `parent` filter (see netbox-community#1684)

* Restored search method on prefix filter

* Extended prefix 'available-ips' endpoint to accept multiple objects (related to netbox-community#1553)

* Standardize on JSON data format for all POST/PUT test client requests

* netbox-community#1694: Initial work on "next available" prefix provisioning

* Closes netbox-community#1706: Added deprecation warning for Python 2

* Merge branch '150-interface-vlans' into develop-2.3

* Fixed up validation of Interface VLAN assignments

* Fixes netbox-community#1645: Simplified interface serialzier for IP addresses and optimized API view queryset

* Cleaned up bulk IP provisioning a bit

* Add Tenancy to Rack Reservations; Fixes netbox-community#1592 (netbox-community#1672)

* fixed prefix header to represent new serial "vlan_vid"

* shows option in creation now

* fixed visibility on rack page

* cleanup

* Added view to Tenant page

* Moved migration for update from netbox-community#1666 and fixed tenant enumeration in FilterForm

* Fixed conflict #1

* Fixed filters from merge and made migration merge

* added tenant to api

* Fixed migrations problem

* Added Tenant to bulkedit option

* Resolved migration collision from netbox-community#1672

* A bit of cosmetic cleanup from netbox-community#1672

* Added nested representations of user and tenant to the rack reservation serializer

* Fixed version number

* Initial work on virtual chassis support

* Moved VC master designation to membership model

* Added initial UI views for virtual chassis assignment

* Fixes netbox-community#1727: Added missing import for M2M_FIELD_TYPES

* Added a form to edit virtual chassis

* Added views for editing/deleting VCMemberships

* Display member interfaces when viewing VC master device

* Added virtual_chassis_id API filter for interfaces

* Obsoleted ComponentEditView and ComponentDeleteView

* Added virtual chassis tests

* Allow designating primary IPs assigned to a device's peer VC members

* Return all VC member interfaces when filtering for the master device; remove virtual_chassis_id filter

* Ignore VC member interfaces where mgmt_only=True

* PEP8 fixes

* Closes netbox-community#1744: Allow associating a platform with a specific manufacturer

* Closes netbox-community#1283: Added a time zone field to the site model

* Fixes netbox-community#1136: Enforce model validation during bulk update

* Closes netbox-community#1321: Added created and last_updated fields for relevant models to their API serializers

* Cleaned up component tables and checkbox toggling

* More table cleanup

* Converted remaining legacy views to class-based views for the DCIM app

* Added bulk creation API tests

* Fix bulk creation of VCMemberships via API

* Fix bulk creation of Secrets via API

* Closes netbox-community#1781: Enable bulk renaming of device components

* Implemented a view for adding individual devices to an existing virtual chassis

* Completed virtual chassis API serializers

* Reflect virtual chassis membership in display_name

* Extend IP address device filter to match virtual chassis members

* Added VirtualChassis to the docs

* Allow interface mode to be null (for routed interfaces)

* Highlight renamed components and allow for multiple previews

* Renamed device status constants for clarity

* Closes netbox-community#1758: Added 'status' field to Site model

* Closes netbox-community#1821: Added 'description' field to Site model

* Removed support for NAPALM 1.x

* Fixes netbox-community#1847: Fix RecursionError when VC master device is unnamed

* Fixes netbox-community#1848: Allow null value for interface encapsulation mode

* netbox-community#1843: Allow assignment of VC member interfaces to VC master LAG

* Fixes netbox-community#1838: Fix KeyError when attempting to create a VirtualChassis with no devicesselected

* Collapsed VCMembership into the Device model (WIP)

* Added virtual chassis member add view

* Added virtual chassis member remove view

* Cleaned up API for virtual chassis

* Added virtual chassis tests

* Additional validation cleanup

* Allow assignment of services to IPs on any VC member

* Closes netbox-community#1864: Added a 'status' field to the circuit model

* Fixes netbox-community#1867: Allow filtering on device status with multiple values

* Upgraded jquery to v3.3.1

* Fix for bulk interface edit form 802.1Q settings (netbox-community#1882)

* fixes netbox-community#1881 - bulk interface 802.1Q settings form

* fix PEP8 newline

* PEP8 fixup

* Fixes netbox-community#1884: Provide additional context to identify devices when creating/editing avirtual chassis

* VirtualChassis form validation cleanup

* Fixed typo in template

* Exclude devices already assigned to a VC from the list of potential VC members

* Finished VirtualChassis list view

* Post-release version bump

* Initial model for Packages model

* Add package_list

* Add migrations

* Adding package add form

* Import CSV Packages feature

* Adding filters

* Working bulkedit functionality for packages

* Require a package for a customer circuit

* Add edit and delete views for packages

* Add API support

* Expose package detail via the API endpoint

* Formatting correction

* Corrected order of arguments on DeviceVCMembershipForm

* Closes netbox-community#1899: Prefer binary package of psycopg2

* Updated requirements list

* Fixes netbox-community#1907: Allow removing an IP as the primary for a device when editing the IP directly

* Release v2.3.0

* NETOPS-704 Rename Tenant Group to Service Providers and Tenants to Customers

* NETOPS-704 Some small detail names fixed

* Fixes
funzoneq added a commit to openfibernet/netbox that referenced this issue Mar 8, 2018
* Closes netbox-community#1553: Introduced support for bulk object creation via the API

* Removed prefix `parent` filter (see netbox-community#1684)

* Restored search method on prefix filter

* Extended prefix 'available-ips' endpoint to accept multiple objects (related to netbox-community#1553)

* Standardize on JSON data format for all POST/PUT test client requests

* netbox-community#1694: Initial work on "next available" prefix provisioning

* Closes netbox-community#1706: Added deprecation warning for Python 2

* Merge branch '150-interface-vlans' into develop-2.3

* Fixed up validation of Interface VLAN assignments

* Fixes netbox-community#1645: Simplified interface serialzier for IP addresses and optimized API view queryset

* Cleaned up bulk IP provisioning a bit

* Add Tenancy to Rack Reservations; Fixes netbox-community#1592 (netbox-community#1672)

* fixed prefix header to represent new serial "vlan_vid"

* shows option in creation now

* fixed visibility on rack page

* cleanup

* Added view to Tenant page

* Moved migration for update from netbox-community#1666 and fixed tenant enumeration in FilterForm

* Fixed conflict #1

* Fixed filters from merge and made migration merge

* added tenant to api

* Fixed migrations problem

* Added Tenant to bulkedit option

* Resolved migration collision from netbox-community#1672

* A bit of cosmetic cleanup from netbox-community#1672

* Added nested representations of user and tenant to the rack reservation serializer

* Fixed version number

* Initial work on virtual chassis support

* Moved VC master designation to membership model

* Added initial UI views for virtual chassis assignment

* Fixes netbox-community#1727: Added missing import for M2M_FIELD_TYPES

* Added a form to edit virtual chassis

* Added views for editing/deleting VCMemberships

* Display member interfaces when viewing VC master device

* Added virtual_chassis_id API filter for interfaces

* Obsoleted ComponentEditView and ComponentDeleteView

* Added virtual chassis tests

* Allow designating primary IPs assigned to a device's peer VC members

* Return all VC member interfaces when filtering for the master device; remove virtual_chassis_id filter

* Ignore VC member interfaces where mgmt_only=True

* PEP8 fixes

* Closes netbox-community#1744: Allow associating a platform with a specific manufacturer

* Closes netbox-community#1283: Added a time zone field to the site model

* Fixes netbox-community#1136: Enforce model validation during bulk update

* Closes netbox-community#1321: Added created and last_updated fields for relevant models to their API serializers

* Cleaned up component tables and checkbox toggling

* More table cleanup

* Converted remaining legacy views to class-based views for the DCIM app

* Added bulk creation API tests

* Fix bulk creation of VCMemberships via API

* Fix bulk creation of Secrets via API

* Closes netbox-community#1781: Enable bulk renaming of device components

* Implemented a view for adding individual devices to an existing virtual chassis

* Completed virtual chassis API serializers

* Reflect virtual chassis membership in display_name

* Extend IP address device filter to match virtual chassis members

* Added VirtualChassis to the docs

* Allow interface mode to be null (for routed interfaces)

* Highlight renamed components and allow for multiple previews

* Renamed device status constants for clarity

* Closes netbox-community#1758: Added 'status' field to Site model

* Closes netbox-community#1821: Added 'description' field to Site model

* Removed support for NAPALM 1.x

* Fixes netbox-community#1847: Fix RecursionError when VC master device is unnamed

* Fixes netbox-community#1848: Allow null value for interface encapsulation mode

* netbox-community#1843: Allow assignment of VC member interfaces to VC master LAG

* Fixes netbox-community#1838: Fix KeyError when attempting to create a VirtualChassis with no devicesselected

* Collapsed VCMembership into the Device model (WIP)

* Added virtual chassis member add view

* Added virtual chassis member remove view

* Cleaned up API for virtual chassis

* Added virtual chassis tests

* Additional validation cleanup

* Allow assignment of services to IPs on any VC member

* Closes netbox-community#1864: Added a 'status' field to the circuit model

* Fixes netbox-community#1867: Allow filtering on device status with multiple values

* Upgraded jquery to v3.3.1

* Fix for bulk interface edit form 802.1Q settings (netbox-community#1882)

* fixes netbox-community#1881 - bulk interface 802.1Q settings form

* fix PEP8 newline

* PEP8 fixup

* Fixes netbox-community#1884: Provide additional context to identify devices when creating/editing avirtual chassis

* VirtualChassis form validation cleanup

* Fixed typo in template

* Exclude devices already assigned to a VC from the list of potential VC members

* Finished VirtualChassis list view

* Post-release version bump

* Formatting correction

* Corrected order of arguments on DeviceVCMembershipForm

* Closes netbox-community#1899: Prefer binary package of psycopg2

* Updated requirements list

* Fixes netbox-community#1907: Allow removing an IP as the primary for a device when editing the IP directly

* Release v2.3.0

* Fixes netbox-community#1915: Redirect to device view after deleting a component

* Fixes netbox-community#1919: Prevent exception when attempting to create a virtual machine without selecting devices

* Closes netbox-community#1918: Add note about copying media directory to upgrade doc

* Fixes netbox-community#1924: Include VID in VLAN lists when editing an interface

* fixed netbox-community#1921 - create interfaces with 801.1q in api

* Post-release version bump (a bit late)

* fixed form bound check for site and vlan group

* Fixes netbox-community#1927: Include all VC member interaces on A side when creating a new interface connection

* Fixes netbox-community#1921: Ignore ManyToManyFields when validating a new object created via the API

* refactor to handle M2M validation in ValidatedModelSerializer

* Fixes netbox-community#1935: Correct API validation of VLANs assigned to interfaces

* Fixes netbox-community#1934: Fixed exception when rendering export template on an object type with custom fields assigned

* Fixes netbox-community#1926: Prevent reassignment of parent device when bulk editing VC member interfaces

* Closes netbox-community#1910: Added filters for cluter group and cluster type

* Fixes netbox-community#1936: Trigger validation error when attempting to create a virtual chassis without specifying member positions

* Release v2.3.1
@lock lock bot locked as resolved and limited conversation to collaborators Jan 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: feature Introduction of new functionality to the application
Projects
None yet
Development

No branches or pull requests

3 participants