Spark based module to perform short-circuit and fuse specificity calculations.
There are two ways to use this module:
- standalone command-line application
- from within the CIMApplication user interface
For both methods, the output, for each node of the network, is an estimate of:
- the network short-circuit power and impedance
- the maximum inrush current (starting current) for a motor
- the list of fuses encountered on the path from the source
- a Go/NoGo flag if the fuse is of the correct size
The stand-alone program is created using Maven:
cd ShortCircuit mvn package
This will produce a jar file in the
target/ directory with a name that indicates the Scala, Spark and program versions.
For example ShortCircuit-2.11-2.3.2-2.4.0-jar-with-dependencies.jar is built for Sala 2.11, Spark 2.3.2 and is version 2.4.0 of the program.
To execute the standalone program use the spark-submit command from within the Spark cluster and specify the CIM file(s) to process:
spark-submit --master spark://sandbox:7077 --conf spark.driver.memory=1g --conf spark.executor.memory=1g ShortCircuit-2.11-2.3.2-2.4.0-jar-with-dependencies.jar --logging WARN hdfs://sandbox:8020/cimfile.rdf
--help option generates a description of the options available:
ShortCircuit 2.11-2.3.2-2.4.0 Usage: ShortCircuit [options] <CIM>,<CIM>... --quiet suppress informational messages [false] --master MASTER_URL spark://host:port, mesos://host:port, yarn, or local[*] --opts k1=v1,k2=v2 Spark options [spark.graphx.pregel.checkpointInterval=8,spark.serializer=org.apache.spark.serializer.KryoSerializer,spark.ui.showConsoleProgress=false] --storage <value> storage level for RDD serialization [MEMORY_AND_DISK_SER] --splitsize <value> file input format maximum size  --deduplicate de-duplicate input (striped) files [false] --logging <value> log level, one of ALL,DEBUG,ERROR,FATAL,INFO,OFF,TRACE,WARN [OFF] --checkpoint <dir> checkpoint directory on HDFS, e.g. hdfs://... --description <text> text describing this program execution for SQLite run table --netp_max <Sk_max> maximum network power if not in CIM, VA [2.00000e+08] --netz_max <r + xj> maximum network impedance if not in CIM, Ω [0.43778578-1.20280655j] --netp_min <Sk_min> minimum network power if not in CIM, VA [1.00000e+08] --netz_min <r + xj> minimum network impedance if not in CIM, Ω [0.43778578-1.20280655j] --tbase <value> temperature assumed in CIM file (°C) [20.0000] --tlow <value> low temperature for maximum fault (°C) [60.0000] --thigh <value> high temperature for minimum fault (°C) [90.0000] --trafos <TRA file> file of transformer names (one per line) to process --trafop <ratedS> transformer power if not in CIM, VA  --trafoz <r + xj> transformer impedance if not in CIM, Ω [0.0059+0.03956248j] --cmax <value> voltage factor for maximum fault level, used for rating equipment [1.00000] --cmin <value> voltage factor for minimum fault level, used for protections settings [0.900000] --cosphi <value> load power factor, used for maximum inrush current [worst case] --fuse_table <value> recommended fuse sizing table, #1 from 65A⇒25 to 2400A⇒630, #2 from 28A⇒10 to 2500A⇒630 , #3 DIN as #1, SEV 200A⇒60 to 1150A⇒400 --messagemax <value> maximum number of warning and error messages per node  --batchsize <value> size of result collections for driver database writes  --cable_impedance_limit <value> cables with higher impedances for R1 will not be processed with gridlabd [5.00000] --workdir <dir> shared directory (HDFS or NFS share) with scheme (hdfs:// or file:/) for work files --help prints this usage text <CIM>,<CIM>... CIM rdf files to process
Together with the --logging option and log4j.properties configuration of Spark, this controls if informational messages from the application are emitted.
Allows specifying the Spark master (same meaning as spark-submit) when
ShortCircuit is run in debug mode in a development environment.
Normally this should not be used by end users.
Within the program, the Spark session is accessed via
so it should be benign to specify the same master as for
Allows specifying Spark options when
ShortCircuit is run.
When specified, options are in the form of
(just like the Arbitrary Spark configuration property
--conf option for
and not the form used by spark-submit.
For example spark-submit uses
--driver-memory 2g, while
Same as specifying
ch.ninecode.cim.split_maxsize=XXX for the CIMReader.
A file larger than this threshold is split into multiple partitions when being read in.
Same as specifying
ch.ninecode.cim.do_deduplication=??? for the CIMReader.
When using striped or tiled RDF files as input, this option will eliminate
duplicated elements contained in multiple files.
Only distinctly identified (rdf:ID) CIM elements are kept, duplicates are discarded.
Specifies the logging level (verbosity) of log records. This overrides the logging level specified in log4j.properties for some ShortCircuit class files.
Specifies the checkpoint directory and turns checkpointing on - if not specified no checkpointing occurs.
An RDD checkpoint is a materialized RDD where the current state of the RDD is stored on disk
and the recipe for how to re-create it is discarded.
have optional checkpoints coded in logic to save the state after costly computations -
which means we trade off time to re-create discarded RDD, for disk space and the time for extra disk/network I/O.
At the moment it is unclear under which conditions checkpointing should be recommended.
NOTE: checkpoint storage must be manually managed (it will not be erased automatically) and this can consume significant space. It is recommended to periodically execute a recursive delete on the checkpoint directory (note the quotes around the asterisk wildcard):
hdfs dfs -rm -R "/checkpoint/*"
User specified text to be included in the
description column of the results database
Simply arbitrary text value describing this program execution.
Enclose the value in quotes to include spaces and funny characters.
netp_max, netz_max, netp_min, netz_min
These four parameters specify the default short circuit characteristics of the medium voltage network. In the absence of a EquivalentInjection element connected to the high voltage node of a transformer, these values are used to set the maximum and minimum available power and upstream impedance. The default values correspond to 200MVA max, 100MVA min at a power factor of -70°.
tbase, tlow, thigh
These three temperature values control the temperature compensation performed by
ShortCircuit for cables.
tbase temperature defines what temperature was used to calculate the impedance values stored within the CIM file.
thigh temperatures specify how the temperature compensation should be done for
maximum fault current and minimum fault current respectively. The former is used in fuse sizing and the latter is
used in fuse specificity.
Allows the specification of a file containing a list of transformers to process. A typical use-case is to analyze one or more problem areas without processing the entire set of transformer service areas.
Set the defaults for transformer power rating and impedance when a transformer does not have these values in the CIM file. The defaults are equivalent to a standard 630kVA old-style oil insulated transformer.
These derating factors are used to approximate the actual voltage at the point of common coupling.
For maximum current (when rating equipment) the
cmax value specifies the voltage factor for maximum fault level,
and is normally set to 1.0 so that the nominal voltage is used.
For minimum current (when checking protections settings) the
cmin value specifies the voltage factor for minimum fault level,
and is normally set to 0.9 so that the simulated voltage is the worst case (lowest voltage).
Specifies the load power factor to be used for a "motor" causing the maximum inrush current. For the worst case scenario this is normally left not specified so that the worst case power factor is used. If a specific power factor is known, the power factor can be specified, overriding the worst case scenario.
Selects the fuse sizing table to be used for determining if a fuse is the correct size.
These tables take the form of a piecewise step approximation between the short circuit current and the fuse rating.
A value of Ik selects the table entry larger than or equal to Ik and retrieves the corresponding fuse rating.
This is compared to the existing fuse rating and if the fuse rating is greater than or equal to the table value it is deemed
For example an Ik of 345A would expect a fuse rating of 125A using table #1.
If the actual fuse is 100A then it is considered
For fuse table #1 the breakpoints are:
For fuse table #2 the breakpoints are:
For fuse table #3, the same fuse table as for #1 is used if the fuse name starts with "DIN". However, if the fuse name starts with "SEV", then the following table is used:
Specifies the maximum number of warning and error messages per node. Messages are encapsulated in a queue that is limited in size to this value.
When storing the results in SQLite, the large number of values must be throttled to avoid database errors. This value sets the batch size - the number of values submitted at one time.
For cables with unknown impedance that have been given an arbitrarily large value, this setting restricts the cable impedance - effectively ignoring (removing) cables with a larger value.
The common directory for working files (GridLAB-D models, players and recorders). This directory should be available to all executors in the cluster so they can perform load-flow analysis in parallel.
Execution from CIMApplication is performed from the Analysis tab:
The program will not provide valid output values for the following cases:
Three Winding Transformers
Where a transformer has more than one winding, the program will not provide valid results for downstream nodes,
and will have a message like
INVALID: 3 transformer windings for edge TRA1234.
Where a transformer has manual or automatic taps to change the voltage ratio, the program will not provide valid results for downstream nodes,
and will have a message like
INVALID: voltage (400V) regulator edge TRA1234.
Where a set of transformers provides a step-up and step-down sequence, the program will not provide valid results for downstream nodes,
and will have a message like
INVALID: low voltage (400V:1000V) subtransmission edge TRA1234.