Tns24 (#63)

* add more info from t2 eval

Updated TNS submit methods.

New plot methods, merged from nuztf.

Updated poetry files.

remove unimplemented method

ruff noqa flags added

chore: ruff cleanup

Remove old TNS units.

clean test

After removing out of day tests, the github ruff test fails (works locally).

Also remove obsolute TNS submit routines.

mypy edits

chore: add ruff to pre-commit config

ruff check

...and correcting ruff changes so mypy still happy.

...and correcting ruff changes so mypy still happy.

type support for Pillow

Plot options for Colibri.

Pull wrt magdifflim as criteria.

Fixed bug in TNS resonse query.

* Allow digest redshift methods to be inherited.

* Activate TNS submission and reply reading.

* added T2KilonovaStats evaluation

* Filter for looking for stellar flares.

* Base unit for lightcurve fitters.

* Two bugs caused by potentially empty catalog entries.

* Units for demo linear fit and search for stellar outburts.

* Updates to new API for TNS and AColibri uploads.

* Remove debug prints.

* Format updates.

* Consistant unit naming and separated base and demo classes.

* rm old names.

* Ensure that a NamedSecret for Slack publishing is not required.

* Shift dependence from TNSMirrorSearcher to complements through TNSNames.

* PlotTransientLightcurves now works with T3 complement cutouts.

* SubmitTNS using complemented TNS ids when possibl.

* Ripped out TNS and cutout duplicated methods.

* refine first-sentence detection

---------

Co-authored-by: andimatter <hermydrea@web.de>
Co-authored-by: Jakob van Santen <jvansanten@gmail.com>

.pre-commit-config.yaml

@@ -1,25 +1,11 @@
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
+
 repos:
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.4.0
-    hooks:
-    -   id: trailing-whitespace
-    -   id: end-of-file-fixer
-    -   id: check-added-large-files
-    -   id: check-json
-        exclude: ampel/contrib/hu/test
-    -   id: check-yaml
-    -   id: pretty-format-json
-        args: [--no-sort-keys, --autofix, --indent=2]
-        exclude: ampel/contrib/hu/test
--   repo: https://github.com/astral-sh/ruff-pre-commit
-    # Ruff version.
-    rev: v0.1.14
-    hooks:
-    # Run the linter.
-    -   id: ruff
-        args: [ --fix ]
-    # Run the formatter.
-    -   id: ruff-format
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: v0.1.14
+  hooks:
+    - id: ruff
+      args: [--fix]
+    - id: ruff-format
 fail_fast: false

README.md

@@ -37,13 +37,15 @@ requires an access token if data is to be retrieved.
 - [RcfFilter](ampel/contrib/hu/t0/RcfFilter.py): Filter for the ZTF Redshift Completeness Factor program..
 - [RedshiftCatalogFilter](ampel/contrib/hu/t0/RedshiftCatalogFilter.py): Filter derived from DecentFilter designed to only accept transients located close to a galaxy in a catalog, and within redshift bounds.
 - [SimpleDecentFilter](ampel/contrib/hu/t0/SimpleDecentFilter.py): General-purpose filter devloped alongside DecentFilter but without use of external catalogs.
+- [StellarFilter](ampel/contrib/hu/t0/StellarFilter.py): a.k.a. as the IndecentFilter, i.e. an inversion of the DecentFilter mainly used for finding extragalactic objects.
 - [TransientInClusterFilter](ampel/contrib/hu/t0/TransientInClusterFilter.py): Filter derived from the DecentFilter, in addition selecting candidates with position compatible with that of nearby galaxy clusters..
 - [XShooterFilter](ampel/contrib/hu/t0/XShooterFilter.py): Filter derived from the DecentFilter, in addition selecting very new transients which are visible from the South.
 
 ### T2 units (augment):
 - [T2BayesianBlocks](ampel/contrib/hu/t2/T2BayesianBlocks.py): T2 unit for running a bayesian block search algorithm to highlight excess regions.
 - [T2BrightSNProb](ampel/contrib/hu/t2/T2BrightSNProb.py): Derive a number of simple metrics describing the rise, peak and decline of a lc.
 - [T2CatalogMatchLocal](ampel/contrib/hu/t2/T2CatalogMatchLocal.py): Cross matches the position of a transient to those of sources in a set of catalogs.
+- [T2DemoLightcurveFitter](ampel/contrib/hu/t2/T2DemoLightcurveFitter.py): Demonstration class showing how methods of T2BaseLightcurveFitter can be used develop a specific classifier.
 - [T2DigestRedshifts](ampel/contrib/hu/t2/T2DigestRedshifts.py): Compare potential matches from different T2 units providing redshifts.
 - [T2DustEchoEval](ampel/contrib/hu/t2/T2DustEchoEval.py)
 - [T2ElasticcRedshiftSampler](ampel/contrib/hu/t2/T2ElasticcRedshiftSampler.py): Parse the elasticc diaSource host information and returns a list of redshifts and weights.
@@ -59,7 +61,7 @@ requires an access token if data is to be retrieved.
 - [T2MatchBTS](ampel/contrib/hu/t2/T2MatchBTS.py): Add information from the BTS explorer page.
 - [T2MultiXgbClassifier](ampel/contrib/hu/t2/T2MultiXgbClassifier.py): For a range of xgboost classifier models, find a classification.
 - [T2NedSNCosmo](ampel/contrib/hu/t2/T2NedSNCosmo.py): Fits lightcurves using SNCOSMO (using SALT2 defaultwise) with redshift constrained by catalog matching results.
-- [T2NedTap](ampel/contrib/hu/t2/T2NedTap.py): See also.
+- [T2NedTap](ampel/contrib/hu/t2/T2NedTap.py): See also: https://ned.ipac.caltech.edu/tap/sync?QUERY=SELECT+*+FROM+TAP_SCHEMA.tables&REQUEST=doQuery&LANG=ADQL&FORMAT=text Export all NED: https://ned.ipac.caltech.edu/tap/sync?QUERY=SELECT+*+FROM+NEDTAP.objdir&REQUEST=doQuery&LANG=ADQL&FORMAT=text.
 - [T2PS1ThumbExtCat](ampel/contrib/hu/t2/T2PS1ThumbExtCat.py): Retrieve panstarrs images at datapoint location and for each tied extcat catalog matching result.
 - [T2PS1ThumbNedSNCosmo](ampel/contrib/hu/t2/T2PS1ThumbNedSNCosmo.py): This state t2 unit is tied with the state T2 unit T2NedSNCosmo.
 - [T2PS1ThumbNedTap](ampel/contrib/hu/t2/T2PS1ThumbNedTap.py): This point t2 unit is tied with the point T2 unit T2NedTap.
@@ -80,14 +82,15 @@ requires an access token if data is to be retrieved.
 - [HealpixCorrPlotter](ampel/contrib/hu/t3/HealpixCorrPlotter.py): Compare healpix coordinate P-value with output from T2RunSncosmo..
 - [HealpixTokenGenerator](ampel/contrib/hu/t3/HealpixTokenGenerator.py): Based on a URL to a Healpix map.
 - [PlotLightcurveSample](ampel/contrib/hu/t3/PlotLightcurveSample.py): Unit plots results from lightcurve fitters (RunSncosmo, RunParsnip).
+- [PlotTransientLightcurves](ampel/contrib/hu/t3/PlotTransientLightcurves.py): Create a (pdf) plot summarizing lightcurves of candidates provided to the unit.
 - [RandomMapGenerator](ampel/contrib/hu/t3/RandomMapGenerator.py): Generate smoothed circular healpix probability values around a random coordinate..
 - [RapidBase](ampel/contrib/hu/t3/RapidBase.py): Trigger rapid reactions.
 - [RapidLco](ampel/contrib/hu/t3/RapidLco.py): Submit LCO triggers for candidates passing criteria..
 - [RapidSedm](ampel/contrib/hu/t3/RapidSedm.py): Select transients for rapid reactions.
 - [ScoreSingleObject](ampel/contrib/hu/t3/ScoreSingleObject.py): Calculate score based on how early a specific SN is detected.
 - [ScoreTNSObjects](ampel/contrib/hu/t3/ScoreTNSObjects.py): Calculate score based on detection time reported to TNS, if any..
 - [SlackSummaryPublisher](ampel/contrib/hu/t3/SlackSummaryPublisher.py)
-- [TNSTalker](ampel/contrib/hu/t3/TNSTalker.py): Get TNS name if existing, and submit selected candidates.
+- [SubmitTNS](ampel/contrib/hu/t3/SubmitTNS.py): Submit candidates to TNS (unless already submitted).
 - [TransientInfoPrinter](ampel/contrib/hu/t3/TransientInfoPrinter.py)
 - [TransientTablePublisher](ampel/contrib/hu/t3/TransientTablePublisher.py): Construct a table based on selected T2 output values.
 - [TransientViewDumper](ampel/contrib/hu/t3/TransientViewDumper.py)