LightMeasurementDevice SOAP interface

[ThE-MaRaC]

SOAP interface for LightMeasurementDevice creation is missing.
It is only possible to assign existing LightMeasurementDevice to Ssld via SetLightMeasurementDeviceRequest from PublicLightingAdHocManagement IF, but not to create it.
Also, AddDeviceRequest from DeviceInstallation IF always creates Ssld.

[kevinsmeets]

We are using some SQL scripts to create a handful of light measurement devices. Those devices have known names, IP addresses etc. There is no need to change/add/remove light measurement devices for our use cases. Coupling SSLD's to LMD's is supported and the coupling can be changed by users.

If you need to add some light measurement devices, I could provide an anonymized version of the SQL scripts. Or if you need the SOAP web service operation(s), the team would be delighted to review your pull request.

[ThE-MaRaC]

SQL script will be appreciated.
Also, I am more than willing to implement LMD SOAP interface if I am allowed to do so and if such feature is acceptable.

[kevinsmeets]

I added this for you: https://github.com/OSGP/Config/tree/development/sql/light-measurement-devices

Please have a look at these basic SQL scripts, these are inserting data which is very much like the production environment uses.

• The names for LMD's and IP addresses etc. have been changed.
• The IP address used is 127.0.0.1 for now.
• The LMD's have device authorizations for the organization test-org.
• The server model / ICD file used can be found here: https://github.com/OSGP/Documentation/blob/development/Protocols/IEC61850/FlexOVL_540_171101_2/FlexOVL_540_171101_2_out.icd.md .

There are psql commands in the shell scripts used to run the SQL scripts. Those shell scripts provide an example for your convenience. Obviously, you can choose how you would like to use or run the queries, I assume you have a tool you prefer to do something with the PostgreSQL databases.

If you would like to use the IEC61850 simulator (https://github.com/OSGP/open-smart-grid-platform/tree/development/osgp/protocol-adapter-iec61850/protocol-simulator-iec61850) to simulate some LMD's, you're going to need some configuration for the protocol adapter and simulator. And a small change to the data as provided by the SQL scripts and a slightly tweaked server model / ICD file. If that's what you want to do, please let me know.

[kevinsmeets]

I think it would be great if you want to contribute some code to this project. The SOAP web service operations to create, update, and delete LMD's are indeed missing. Even though we are using those SQL scripts, it would be nice if the platform web service supports those operations.

[ThE-MaRaC]

Here is what I have managed to implement so far.
DeviceInstallation IF has been extended with 2 operations:

• AddLightMeasurementDevice
• UpdateLightMeasurementDevice

The following DeviceManagement IF operations can be resued:

• FindDevices
• RemoveDevice

The following PublicLightingAdHocManagement IF operations can also be resued:

• FindAllDevices
• SetLightMeasurementDevice

Additionally, I plan to add a few integration tests, but if you want I can provide a pull request only with source code and update it when tests are finished.

[kevinsmeets]

That sounds very nice, @ThE-MaRaC. I think it would be good to create a pull request with the initial changes. That way, you can have a look at the CLA and decide if you are in agreement with it. I don't mind a pull request which is updated at a later time.

[ThE-MaRaC]

Pull request is available: OSGP/open-smart-grid-platform#504
At the moment there are only 2 integration tests, but I plan to add a few more.

[ThE-MaRaC]

LMD SOAP Interface integration tests are updated.

Regression (cucumber-tests-platform-common) results
420 Scenarios (420 passed)
1394 Steps (1394 passed)
3m53.974s

[ThE-MaRaC]

Additionally, I noticed that none of the currently available FindDevices/FindAllDevices operations is able to return LMD data.
So, my proposal is to extend FindAllDevices from PublicLightingAdHocManagement IF to return LMD's and SSLD's.
Something like this:

  <xsd:complexType name="Ssld">
    <xsd:complexContent>
      <xsd:extension base="tns:Device">
        <xsd:sequence>
          <xsd:element name="HasSchedule" type="xsd:boolean" />
          <xsd:element name="PublicKeyPresent" type="xsd:boolean" />
        </xsd:sequence>
      </xsd:extension>
    </xsd:complexContent>
  </xsd:complexType>

  <xsd:complexType name="LightMeasurementDevice">
    <xsd:complexContent>
      <xsd:extension base="tns:Device">
        <xsd:sequence>
          <xsd:element name="Description" type="xsd:string" minOccurs="0" />
          <xsd:element name="Code" type="xsd:string" minOccurs="0" />
          <xsd:element name="Color" type="xsd:string" minOccurs="0" />
          <xsd:element name="DigitalInput" type="xsd:short" minOccurs="0" />
        </xsd:sequence>
      </xsd:extension>
    </xsd:complexContent>
  </xsd:complexType>

Another option would be to implement a completely new operation which will return only LMD's.
What do you think? Do you prefer any solution?

[kevinsmeets]

The operation FindDevices (https://github.com/OSGP/open-smart-grid-platform/blob/3295edbd4f35b3f8243041e5dcd1f9da31ef5be1/osgp/shared/osgp-ws-core/src/main/resources/DeviceManagement.wsdl#L252) part of the core web service component returns all devices. See the endpoint function org.opensmartgridplatform.adapter.ws.core.endpoints.DeviceManagementEndpoint.findDevices(String, FindDevicesRequest). The list of returned device does depend on organization and device authorizations. This is some logging of such a request, the response is quite large so I left it out:

2020-11-18 22:09:23.572] [dev-box] [ajp-nio-8009-exec-2] INFO o.o.a.w.e.WebServiceMonitorInterceptor@printSoapMessage:338 - soap message: 
<SOAP-ENV:Envelope
	xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
	<SOAP-ENV:Header>
		<OrganisationIdentification
			xmlns="http://www.opensmartgridplatform.org/schemas/common">some-organization
		</OrganisationIdentification>
		<ApplicationName
			xmlns="http://www.opensmartgridplatform.org/schemas/common">some-application
		</ApplicationName>
		<UserName
			xmlns="http://www.opensmartgridplatform.org/schemas/common">some-user
		</UserName>
	</SOAP-ENV:Header>
	<SOAP-ENV:Body>
		<ns2:FindDevicesRequest
			xmlns:ns2="http://www.opensmartgridplatform.org/schemas/devicemanagement/2014/10"
			xmlns:ns3="http://www.opensmartgridplatform.org/schemas/common/2014/10">
			<ns2:PageSize>50</ns2:PageSize>
			<ns2:Page>0</ns2:Page>
			<ns2:DeviceFilter>
				<ns2:DeviceActivated>ACTIVE</ns2:DeviceActivated>
				<ns2:SortDir>asc</ns2:SortDir>
				<ns2:SortedBy>deviceIdentification</ns2:SortedBy>
				<ns2:HasTechnicalInstallation>false</ns2:HasTechnicalInstallation>
			</ns2:DeviceFilter>
			<ns2:UsePages>true</ns2:UsePages>
		</ns2:FindDevicesRequest>
	</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

2020-11-18 22:09:23.573] [dev-box] [ajp-nio-8009-exec-2] INFO o.o.a.w.c.e.DeviceManagementEndpoint@findDevices:290 - Find devices for organisation: some-organization.
2020-11-18 22:09:23.576] [dev-box] [ajp-nio-8009-exec-2] INFO o.o.d.core.services.SecurityService@checkAuthorization:84 - Organisation some-organization is allowed function FIND_DEVICES
2020-11-18 22:09:24.173] [dev-box] [ajp-nio-8009-exec-2] INFO o.o.a.w.e.WebServiceMonitorInterceptor@printSoapMessage:338 - soap message: ... returns LMD's and SSLD's ...

I don't think there's a PublicLighting web service operation that is returning LMD data. I would have to look into the actual usage of the operation(s) for retrieving devices as part of the PublicLighting web service. Personally, I think it would be nice if the PublicLighting web service could be used to retrieve both LMD's and SSLD's.

[ThE-MaRaC]

PublicLightingAdHocManagement FindAllDevices operation has been extended to return SSLD's and LMD's.

Regression (cucumber-tests-platform-publiclighting) results
441 Scenarios (441 passed)
2155 Steps (2155 passed)
14m31.235s

[Rupali-2]

I added this for you: https://github.com/OSGP/Config/tree/development/sql/light-measurement-devices

Please have a look at these basic SQL scripts, these are inserting data which is very much like the production environment uses.

• The names for LMD's and IP addresses etc. have been changed.
• The IP address used is 127.0.0.1 for now.
• The LMD's have device authorizations for the organization test-org.
• The server model / ICD file used can be found here: https://github.com/OSGP/Documentation/blob/development/Protocols/IEC61850/FlexOVL_540_171101_2/FlexOVL_540_171101_2_out.icd.md .

There are psql commands in the shell scripts used to run the SQL scripts. Those shell scripts provide an example for your convenience. Obviously, you can choose how you would like to use or run the queries, I assume you have a tool you prefer to do something with the PostgreSQL dat abases.

If you would like to use the IEC61850 simulator (https://github.com/OSGP/open-smart-grid-platform/tree/development/osgp/protocol-adapter-iec61850/protocol-simulator-iec61850) to simulate some LMD's, you're going to need some configuration for the protocol adapter and simulator. And a small change to the data as provided by the SQL scripts and a slightly tweaked server model / ICD file. If that's what you want to do, please let me know.

Hi ThE-MaRaC,
We are testing IEC61850 simulator. We have seen the communication is not happening between adaptor and simulator. Could you please elaborate more on what needs to be change in configuration and IED files. Also any idea how IEC devices gets registered on osgp platform as we did't find RegisterDevice call from simulator.

Thanks for reading through, your help will be greatly appreciated.

[ThE-MaRaC]

@Rupali-2 I am not using protocol-simulator-iec61850, instead, I have Raspberry Pi + Photoresistor. On Pi I run MMS server which is written in https://libiec61850.com/libiec61850.
RegisterDevice is not part of the IEC61850 protocol, instead, you should send TCP request to port 5000 (iec61850.port.listener). Check org.opensmartgridplatform.adapter.protocol.iec61850.infra.networking.Iec61850ChannelHandlerServer.processRegistrationMessage implementation. As I can see from the source code, protocol-simulator-iec61850 doesn't support RegisterDevice request.

[bvdzwet]

The changes to add, update and request light measurement devices using the public lighting web service operations from pull requests OSGP/open-smart-grid-platform#504 and OSGP/open-smart-grid-platform#534 have been merged.