Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
The are 3 main features introduced within this release:
The full use of the new version of the Request Management System, introduced first within release v6r9, together with the Transformation System and Data Management system.
Many changes are applied in the TransformationSystem, and particularly in the TransformationDB.
A new "package", named Workflow, has been introduced. This package makes it easier, for VOs, to define Job Workflows, introducing generic modules implementations that can be extended by the VOs.
Those new features that need special care when upgrading from previous versions to v6r10 are outlined below.
New RMS everywhere
Using the new RMS system everywhere means using it for all the replication activities, that were using the old RMS system up to version v6r9. The old system, populating the old set of tables, will not be feed any more. System administrators might want to check that all the requests are treated within the old system before stopping the corresponding agents: this is the reason why, in v6r10 code (as well as v6r9), a mix of services and agents from the old and new system are present. The code treating the old system will be removed from the next minor, or major, release of DIRAC. If you are upgrading from DIRAC v6r9, no new agents nor services need to be started.
Changes have been applied in several places, including Accounting and Transformation System.
As part of the changes for the new RMS, a new DB, and a new set of tables will be created at the start of the FTSManagerService, provided that the FTSDB has been previously created. This DB, together with the new implementation of the service and clients, will substitute the old system. Old requests present in the old system will keep to be treated by the old services and agents.
Apply the following:
ALTER Table Operation MODIFY Arguments MEDIUMBLOB;
Changes at the DB level to clean up indexes and speed up queries. The following changes could be done in the mysql database independently from the release installed:
drop of duplicate indexes on
drop of duplicate indexes on
creation of multi-field index on
UniqueIndex). Fields are reordered wrt
UniqueConstraintindex, beginning with
drop of (all) unused single-field indexes on
drop of less performant multi-field index on
change the following record::
update ac_catalog_Types set keyFields='Status,Site,User,UserGroup,JobGroup,MinorStatus,ApplicationStatus,JobSplitType' where name='Dirac-Production_WMSHistory';
here DIRAC-Production is the DIRAC setup. Do that for all the setups.
As an example, here is the SQL for WMSHistory:
CREATE UNIQUE INDEX UniqueIndex ON
Please note: create and drop index operations may take from minutes to several hours (even days), depending on table size
Few code changes are:
- the TransformationClient now provides an easily extensible State Machine for Transformation Status and for the status of the Transformation Files.
- when deriving a transformation, the Transformation Files status will be deterministic.
- the methods for the Replicas table have been deleted. But the biggest changes are at the DB level: please read further.
The tables definition is switching from MyISAM to InnoDB, and few triggers are being introduced. The code will be fully compatible, so there is no need, for any VOs, to switch MyISAM to InnoDB. It is anyway highly recommended. Applying this change needs special care. It has to be noted that, for this release, both the tables definition, and the triggers, are defined in the TransformationDB.sql file, and that system administrators are required to manually intervene at the database level to apply them. The changes to apply follow:
- TransformationDB.Replicas can be fully removed, including its content.
- The other changes are coded in https://github.com/DIRACGrid/DIRAC/pull/1731/files, and includes settings of PKs, FKs, and a trigger. Pay special attention to the TransformationFiles and TransformationTasks table.
In any case, to repeat: VOs can schedule the change of the schema at their favourite time, independently from the introduction of DIRAC v6r10, because tables defined within MyISAM will keep working.
After the tables were changed, restart the TransformationManager.
The new workflow package provides four generic modules. Detailed instructions on how to use it and extend it can be found in the DIRAC.Workflow.Modules.ModuleBase module.
Some new policies have been introduced and are ready to be used. The configuration of the authorization should be changed from "get = all" to "select = all".
- A small change has been applied to the JobLoggingDB: the primary key changed, as well as a new Trigger has been introduced. Details can be seen in https://github.com/DIRACGrid/DIRAC/pull/1724/files
- A new JobPolicy is in place <>