Python: Modernize 4 queries for missing/multiple calls to init/del methods

[joefarebrother]

Modernizes py/missing-call-to-init, py/missing-call-to-del, py/multiple-calls-to-init,, and py/multiplecalls-to-del.
Uses DataFlowDispatch methods rather than. pointsTo.

[github-actions]

QHelp previews:

python/ql/src/Classes/CallsToInitDel/MissingCallToDel.qhelp

Missing call to superclass __del__ during object destruction

Python, unlike some other object-oriented languages such as Java, allows the developer complete freedom in when and how superclass finalizers are called during object finalization. However, the developer has responsibility for ensuring that objects are properly cleaned up, and that all superclass __del__ methods are called.

Classes with a __del__ method (a finalizer) typically hold some resource such as a file handle that needs to be cleaned up. If the __del__ method of a superclass is not called during object finalization, it is likely that resources may be leaked.

A call to the __init__ method of a superclass during object initialization may be unintentionally skipped:

• If a subclass calls the __del__ method of the wrong class.
• If a call to the __del__ method of one its base classes is omitted.
• If a call to super().__del__ is used, but not all __del__ methods in the Method Resolution Order (MRO) chain themselves call super(). This in particular arises more often in cases of multiple inheritance.

Recommendation

Ensure that all superclass __del__ methods are properly called. Either each base class's finalize method should be explicitly called, or super() calls should be consistently used throughout the inheritance hierarchy.

Example

In the following example, explicit calls to __del__ are used, but SportsCar erroneously calls Vehicle.__del__. This is fixed in FixedSportsCar by calling Car.__del__.

class Vehicle(object):
    
    def __del__(self):
        recycle(self.base_parts)
        
class Car(Vehicle):
    
    def __del__(self):
        recycle(self.car_parts)
        Vehicle.__del__(self)
        
#BAD: Car.__del__ is not called.
class SportsCar(Car, Vehicle):
    
    def __del__(self):
        recycle(self.sports_car_parts)
        Vehicle.__del__(self)
        
#GOOD: Car.__del__ is called correctly.
class FixedSportsCar(Car, Vehicle):
    
    def __del__(self):
        recycle(self.sports_car_parts)
        Car.__del__(self)
        

References

• Python Reference: del.
• Python Standard Library: super.
• Python Glossary: Method resolution order.

python/ql/src/Classes/CallsToInitDel/MissingCallToInit.qhelp

Missing call to superclass __init__ during object initialization

Python, unlike some other object-oriented languages such as Java, allows the developer complete freedom in when and how superclass initializers are called during object initialization. However, the developer has responsibility for ensuring that objects are properly initialized, and that all superclass __init__ methods are called.

If the __init__ method of a superclass is not called during object initialization, this can lead to errors due to the object not being fully initialized, such as having missing attributes.

A call to the __init__ method of a superclass during object initialization may be unintentionally skipped:

• If a subclass calls the __init__ method of the wrong class.
• If a call to the __init__ method of one its base classes is omitted.
• If a call to super().__init__ is used, but not all __init__ methods in the Method Resolution Order (MRO) chain themselves call super(). This in particular arises more often in cases of multiple inheritance.

Recommendation

Ensure that all superclass __init__ methods are properly called. Either each base class's initialize method should be explicitly called, or super() calls should be consistently used throughout the inheritance hierarchy.

Example

In the following example, explicit calls to __init__ are used, but SportsCar erroneously calls Vehicle.__init__. This is fixed in FixedSportsCar by calling Car.__init__.

class Vehicle(object):
    
    def __init__(self):
        self.mobile = True
        
class Car(Vehicle):
    
    def __init__(self):
        Vehicle.__init__(self)
        self.car_init()
        
# BAD: Car.__init__ is not called.
class SportsCar(Car, Vehicle):
    
    def __init__(self):
        Vehicle.__init__(self)
        self.sports_car_init()
        
# GOOD: Car.__init__ is called correctly.
class FixedSportsCar(Car, Vehicle):
    
    def __init__(self):
        Car.__init__(self)
        self.sports_car_init()
        

References

• Python Reference: init.
• Python Standard Library: super.
• Python Glossary: Method resolution order.

python/ql/src/Classes/CallsToInitDel/SuperclassDelCalledMultipleTimes.qhelp

Multiple calls to __del__ during object destruction

Python, unlike some other object-oriented languages such as Java, allows the developer complete freedom in when and how superclass finalizers are called during object finalization. However, the developer has responsibility for ensuring that objects are properly cleaned up.

Objects with a __del__ method (a finalizer) often hold resources such as file handles that need to be cleaned up. If a superclass finalizer is called multiple times, this may lead to errors such as closing an already closed file, and lead to objects not being cleaned up properly as expected.

There are a number of ways that a __del__ method may be be called more than once.

• There may be more than one explicit call to the method in the hierarchy of __del__ methods.
• In situations involving multiple inheritance, an finalization method may call the finalizers of each of its base types, which themselves both call the finalizer of a shared base type. (This is an example of the Diamond Inheritance problem)
• Another situation involving multiple inheritance arises when a subclass calls the __del__ methods of each of its base classes, one of which calls super().__del__. This super call resolves to the next class in the Method Resolution Order (MRO) of the subclass, which may be another base class that already has its initializer explicitly called.

Recommendation

Ensure that each finalizer method is called exactly once during finalization. This can be ensured by calling super().__del__ for each finalizer method in the inheritance chain.

Example

In the following example, there is a mixture of explicit calls to __del__ and calls using super(), resulting in Vehicle.__del__ being called twice. FixedSportsCar.__del__ fixes this by using super() consistently with the other delete methods.

#Calling a method multiple times by using explicit calls when a base uses super()
class Vehicle(object):
     
    def __del__(self):
        recycle(self.base_parts)
        super(Vehicle, self).__del__()
        
class Car(Vehicle):
    
    def __del__(self):
        recycle(self.car_parts)
        super(Car, self).__del__()
        
        
class SportsCar(Car, Vehicle):
    
    # BAD: Vehicle.__del__ will get called twice
    def __del__(self):
        recycle(self.sports_car_parts)
        Car.__del__(self)
        Vehicle.__del__(self)
        
        
# GOOD: super() is used ensuring each del method is called once.
class FixedSportsCar(Car, Vehicle):
    
    def __del__(self):
        recycle(self.sports_car_parts)
        super(SportsCar, self).__del__()
     

References

• Python Reference: del.
• Python Standard Library: super.
• Python Glossary: Method resolution order.
• Wikipedia: The Diamond Problem.

python/ql/src/Classes/CallsToInitDel/SuperclassInitCalledMultipleTimes.qhelp

Multiple calls to __init__ during object initialization

Python, unlike some other object-oriented languages such as Java, allows the developer complete freedom in when and how superclass initializers are called during object initialization. However, the developer has responsibility for ensuring that objects are properly initialized.

Calling an __init__ method more than once during object initialization risks the object being incorrectly initialized, as the method and the rest of the inheritance chain may not have been written with the expectation that it could be called multiple times. For example, it may set attributes to a default value in a way that unexpectedly overwrites values setting those attributes in a subclass.

There are a number of ways that an __init__ method may be be called more than once.

• There may be more than one explicit call to the method in the hierarchy of __init__ methods.
• In situations involving multiple inheritance, an initialization method may call the initializers of each of its base types, which themselves both call the initializer of a shared base type. (This is an example of the Diamond Inheritance problem)
• Another situation involving multiple inheritance arises when a subclass calls the __init__ methods of each of its base classes, one of which calls super().__init__. This super call resolves to the next class in the Method Resolution Order (MRO) of the subclass, which may be another base class that already has its initializer explicitly called.

Recommendation

Take care whenever possible not to call an an initializer multiple times. If each __init__ method in the hierarchy calls super().__init__(), then each initializer will be called exactly once according to the MRO of the subclass. When explicitly calling base class initializers (such as to pass different arguments to different initializers), ensure this is done consistently throughout, rather than using super() calls in the base classes.

In some cases, it may not be possible to avoid calling a base initializer multiple times without significant refactoring. In this case, carefully check that the initializer does not interfere with subclass initializers when called multiple times (such as by overwriting attributes), and ensure this behavior is documented.

Example

In the following (BAD) example, the class D calls B.__init__ and C.__init__, which each call A.__init__. This results in self.state being set to None as A.__init__ is called again after B.__init__ had finished. This may lead to unexpected results.

class A:
    def __init__(self):
        self.state = None 

class B(A):
    def __init__(self):
        A.__init__(self)
        self.state = "B"
        self.b = 3 

class C(A):
    def __init__(self):
        A.__init__(self)
        self.c = 2 

class D(B,C):
    def __init__(self):
        B.__init__(self)
        C.__init__(self) # BAD: This calls A.__init__ a second time, setting self.state to None.
        

In the following (GOOD) example, a call to super().__init__ is made in each class in the inheritance hierarchy, ensuring each initializer is called exactly once.

class A:
    def __init__(self):
        self.state = None 

class B(A):
    def __init__(self):
        super().__init__()
        self.state = "B"
        self.b = 3 

class C(A):
    def __init__(self):
        super().__init__()
        self.c = 2 

class D(B,C):
    def __init__(self): # GOOD: Each method calls super, so each init method runs once. self.stat =e will be set to "B".
        super().__init__()
        self.d = 1
        

In the following (BAD) example, explicit base class calls are mixed with super() calls, and C.__init__ is called twice.

class A:
    def __init__(self):
        print("A")
        self.state = None 

class B(A):
    def __init__(self):
        print("B")
        super().__init__() # When called from D, this calls C.__init__
        self.state = "B"
        self.b = 3 

class C(A):
    def __init__(self):
        print("C")
        super().__init__()
        self.c = 2 

class D(B,C):
    def __init__(self): 
        B.__init__(self)
        C.__init__(self) # BAD: C.__init__ is called a second time

References

• Python Reference: init.
• Python Standard Library: super.
• Python Glossary: Method resolution order.
• Wikipedia: The Diamond Problem.

[joefarebrother]

I believe this is a little more readable, conveying the intent that we're looking for cases where no class can be tracked here.

[intrigus-lgtm]

Isn't using toString in queries discouraged?

[joefarebrother]

That is true.
Probably best to compare the locations start lines/columns directly.
Have updated.