How can we support fighting Sirens?

[sukitpr]

For most of events, there are some sirens spawn to the map.

• Code doesn't recognize and not try to calculate them yet.
• The sirens graphic is quite big target which conceal other normal enemy fleet then the code cannot get target enemy location

So I try to add support them at below code in combat.py:

ALAuto/modules/combat.py

Lines 442 to 460 in 70f52a5

	while self.l == []:
	Utils.update_screen()
	l1 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] - 3, x[1] - 45], Utils.find_all('enemy/fleet_level', sim - 0.15, self.chapter_map)))
	l1 = [x for x in l1 if (not self.filter_blacklist(x, blacklist))]
	l2 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_1_down', sim, self.chapter_map)))
	l2 = [x for x in l2 if (not self.filter_blacklist(x, blacklist))]
	l3 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_2_down', sim - 0.02, self.chapter_map)))
	l3 = [x for x in l3 if (not self.filter_blacklist(x, blacklist))]
	l4 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 130], Utils.find_all('enemy/fleet_3_up', sim - 0.06, self.chapter_map)))
	l4 = [x for x in l4 if (not self.filter_blacklist(x, blacklist))]
	l5 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_3_down', sim - 0.06, self.chapter_map)))
	l5 = [x for x in l5 if (not self.filter_blacklist(x, blacklist))]
	self.l = l1 + l2 + l3 + l4 + l5
	sim -= 0.005
	self.l = Utils.filter_similar_coords(self.l)
	return self.l

I added new l6, l7, l8 by trying to calculate from graphic from enemy sirens of each type (DD, CL, CA) but couldn't make it.

I'm quite new in python so please shade me some light so I can open PR later :)

[Egoistically]

I'd like to see your code and the assets you used.

[sukitpr]

Sure

    def get_enemies(self, blacklist=[]):
        sim = 0.99
        if blacklist != []:
            Logger.log_info('Blacklist: ' + str(blacklist))
            self.l = [x for x in self.l if (x not in blacklist)]

        while self.l == []:
            Utils.update_screen()

            l1 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] - 3, x[1] - 45], Utils.find_all('enemy/fleet_level', sim - 0.15, self.chapter_map)))
            l1 = [x for x in l1 if (not self.filter_blacklist(x, blacklist))]
            l2 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_1_down', sim, self.chapter_map)))
            l2 = [x for x in l2 if (not self.filter_blacklist(x, blacklist))]
            l3 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_2_down', sim - 0.02, self.chapter_map)))
            l3 = [x for x in l3 if (not self.filter_blacklist(x, blacklist))]
            l4 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 130], Utils.find_all('enemy/fleet_3_up', sim - 0.06, self.chapter_map)))
            l4 = [x for x in l4 if (not self.filter_blacklist(x, blacklist))]
            l5 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 75, x[1] + 110], Utils.find_all('enemy/fleet_3_down', sim - 0.06, self.chapter_map)))
            l5 = [x for x in l5 if (not self.filter_blacklist(x, blacklist))]
            l6 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 35, x[1] + 100], Utils.find_all('enemy/fleet_siren_dd_blue', sim - 0.1, self.chapter_map)))
            l6 = [x for x in l6 if (not self.filter_blacklist(x, blacklist))]
            l7 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 35, x[1] + 100], Utils.find_all('enemy/fleet_siren_ca_blue', sim - 0.1, self.chapter_map)))
            l7 = [x for x in l7 if (not self.filter_blacklist(x, blacklist))]
            l8 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 35, x[1] + 100], Utils.find_all('enemy/fleet_siren_cl_blue', sim - 0.1, self.chapter_map)))
            l8 = [x for x in l8 if (not self.filter_blacklist(x, blacklist))]
            l9 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 35, x[1] + 100], Utils.find_all('enemy/fleet_siren_cv_blue', sim - 0.1, self.chapter_map)))
            l9 = [x for x in l9 if (not self.filter_blacklist(x, blacklist))]

            self.l = l1 + l2 + l3 + l4 + l5 + l6 + l7 + l8 + l9
            sim -= 0.005

        self.l = Utils.filter_similar_coords(self.l)
        return self.l

The assets I used are

When I run it, it crash with below msg

Traceback (most recent call last):
  File ".\ALAuto.py", line 213, in <module>
    script.run_sortie_cycle()
  File ".\ALAuto.py", line 79, in run_sortie_cycle
    self.run_combat_cycle()
  File ".\ALAuto.py", line 87, in run_combat_cycle
    result = self.modules['combat'].combat_logic_wrapper()
  File "C:\Users\XXX\Documents\git\ALAuto\modules\combat.py", line 88, in combat_logic_wrapper
    if not self.clear_map():
  File "C:\Users\XXX\Documents\git\ALAuto\modules\combat.py", line 370, in clear_map
    target_info = self.get_closest_target(self.blacklist)
  File "C:\Users\XXX\Documents\git\ALAuto\modules\combat.py", line 535, in get_closest_target
    enemies = self.get_enemies(blacklist)
  File "C:\Users\XXX\Documents\git\ALAuto\modules\combat.py", line 455, in get_enemies
    l6 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] + 35, x[1] + 100], Utils.find_all('enemy/fleet_siren_dd_blue', sim - 0.1, self.chapter_map)))
  File "C:\Users\XXX\Documents\git\ALAuto\util\utils.py", line 216, in find_all
    match = cv2.matchTemplate(screen, template, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(3.4.4) C:\projects\opencv-python\opencv\modules\imgproc\src\templmatch.cpp:589: error: (-215:Assertion failed) corrsize.height <= img.rows + templ.rows - 1 && corrsize.width <= img.cols + templ.cols - 1 in function 'cv::crossCorr'

[hjk22]

I pasted your code and your assets in my working tree: the code ran fine, I didn't get your error. I think you might have named your assets in a different way or put them in another folder and cv2 failed to load them. As for the assets, I think they need to be remade since even after lowering the similarity threshold to below 0.9 they weren't detected and the bot started findings random points on the map. Did you get the screenshots by running adb exec-out screencap -p > screenshot.png? Did you do anything else beside cropping the image? I'm not sure, but they could have been modified when you uploaded them.

[Egoistically]

Here you have a more detailed guide: https://github.com/Egoistically/ALAuto/wiki/Creating-new-assets-for-bot.

[didzkows]

Referencing this since I'd like to make the siren addition to my own working copy, the code above should still work in the current latest version correct? It's just potentially a matter of remaking the assets?

[Egoistically]

Yeah with proper assets it should work fine.

[sukitpr]

I pasted your code and your assets in my working tree: the code ran fine, I didn't get your error. I think you might have named your assets in a different way or put them in another folder and cv2 failed to load them. As for the assets, I think they need to be remade since even after lowering the similarity threshold to below 0.9 they weren't detected and the bot started findings random points on the map. Did you get the screenshots by running adb exec-out screencap -p > screenshot.png? Did you do anything else beside cropping the image? I'm not sure, but they could have been modified when you uploaded them.

Did you get the screenshots by running adb exec-out screencap -p > screenshot.png?

Yes, I did.

Did you do anything else beside cropping the image?

I cropped via ACDSee, save it, then put it in /assets/enemy/

I'm not sure, but they could have been modified when you uploaded them.

Thank you @hjk22, @Egoistically
I'll take a look into this more. Or will try to make them again :)

[sukitpr]

I able to support fighting sirens now.
But somehow, code detected some coordinate as enemy while there's no any enemy.
I'll take a look more to fix this issue (high potential from my code). Then I'll try open PR to support Sirens with configuration to on/off.

[didzkows]

Gonna give it a go with these assets, also have a question regarding assets: if you clear the background out and leave it transparent keeping only the actual siren's sprite) would detection be more accurate?

in this case for example, removing the background to leave just parts of the siren

regarding the code in combat.py ,
For example this portion here

ALAuto/modules/combat.py

Line 445 in 70f52a5

l1 = filter(lambda x:x[1] > 80 and x[1] < 977 and x[0] > 180, map(lambda x:[x[0] - 3, x[1] - 45], Utils.find_all('enemy/fleet_level', sim - 0.15, self.chapter_map)))

what does the -0.15 signify? what does altering the value do, as i noticed it is different (in the following lines) based on what asset the script is looking for

For the moment everything seems to work fine, script found sirens and engaged them correctly

Think it's safe to say that everything is working smoothly, left the script unattended while running some errands and there were no problems

[sukitpr]

@didzkows, I think that's the offset for param similarity of function Utils.find_all. We offset it for the most likely of matching algorithm of OpenCV

[Egoistically]

What @sukitpr said, some enemies are easier/harder to detect than others, that offsets default similarity.

[didzkows]

@Egoistically Cheers, any input on my question regarding assets.

if you clear the background out and leave it transparent keeping only the actual siren's sprite) would detection be more accurate?

[sukitpr]

I haven't tried that yet.
But afaik, Bitmap comparing is not ignoring the transparent,
opencv would get transparent bitmap != non-transparent bitmap.

Thus, I think transparent wouldn't work.
But my thought night be out of date as well.

You could give it a try! :)

[hjk22]

I'll try answering your question about the background transparency in assets @didzkows, but I'm not an expert of image recognition algorithms.
As far as I know, the matchTemplate function we use does not handle transparent pixels the way you would want them to be used, i.e. simply ignoring them when searching for match, but they are used during comparisons as well. So I don't think using these types of assets would automatically increase accuracy.
But, since OpenCV 3 (the one we are using for the bot), the matchTemplate function supports the use of masks, allowing you to ignore the pixels you don't want to be compared. To use masks in our code we would need to change the comparison method currently used though, and I don't think we will work on this anytime soon, since they are not indispensable right now.
If you are interesed in this topic I can link you some resources related to it (opencv documentation and stackoverflow questions).
Template matching tutorial
matchTemplate()
Template matching with masks
OpenCV template matching and transparency
Matching template images with masks using OpenCV and C++
How to find a template in an image using a mask (or transparency) with OpenCV and Python?

EDIT: I didn't see your answer sukitpr, but it's pretty much what you said.

[didzkows]

Thank you kindly, looks fascinating but as you mentioned not something that is overly important with regards to this project.

[hjk22]

Closing this issue since an option to detect sirens has been implemented.