Bot should attempt next target on list when first target does not work. [proposed label] application-eve-online

[zoom-maestro]

I found that the bot already does this in the following example:

The bot selected the second item in the overview list when it noticed that the first one was not valid.

Can we do the same for this one?

Several ways to approach this exception:

• Notice the difference in color; thereby decide to target the next valid item on the overview list list.
• Notice that clicking on this item takes more than 3 seconds to bring up the next step (the menu to lock target); thereby decide to target the next valid item on the overview list.

[zoom-maestro]

@Viir Would it be possible to put a "check" on this function to test if a menu item does not appear after 2-3 seconds of the useContextMenuCascadeOnOverviewEntry call, that it moves the mouse "home" and then either attempts again (or possibly does a fallback to normal behavior; as the moving of the mouse should unlock the menu)

[Viir]

@Viir Would it be possible to put a "check" on this function to test if a menu item does not appear after 2-3 seconds of the useContextMenuCascadeOnOverviewEntry call, that it moves the mouse "home" and then either attempts again (or possibly does a fallback to normal behavior; as the moving of the mouse should unlock the menu)

I am not sure I understand the whole sentence.
This section of the guide on developing for EVE Online shows how it works in the popular framework for EVE Online: https://to.botengine.org/guide/developing-for-eve-online#data-flow-for-a-bot-step
It also contains this diagram to clarify the data flow around your function:

To make the data available in the function that you highlighted, you put it in the app memory.

As far as I understand your scenario, it seems following approach is simpler:

Notice the difference in color; thereby decide to target the next valid item on the overview list list.

[zoom-maestro]

I appreciate your non-understanding of my entire sentence. I hope you also appreciate that I have no idea how to read that diagram. As such, my sentence was a result from staring at the the .ELM code over the span of 2-3 days and finally having an "A-HA!" moment.

Going directly to your last statement:

As far as I understand your scenario, it seems following approach is simpler:

Notice the difference in color; thereby decide to target the next valid item on the overview list list.

I agree with you wholeheartedly. So, I think it's best that we focus on that part.

While I observe how the bot selects a target, it appears that it doesn't just pick the TOPMOST target in the Overview and then carry out the "locking" procedure. There is a decision being made:

• It will skip over other player ships to find the appropriate target.
• It will skip over rats to find the appropriate target.
• It will skip over belts to find the appropriate target.

I'm not sure in the code, where it is conducting this decision making. (And this is probably where I make the mistake of wanting to go around the deeper code and implement a more complex solution; as stated in my non-understood question :) )

As such:

How do we add this difference in color target to the "Skip Over" list or remove it from the "Do Not Skip Over list" that is somewhere deeper in the code?

[Viir]

That depends on which code you used to arrive at your observation in the first place. Which app did you use?
I did not find a mention of the app ID or a link to your program code.

[zoom-maestro]

That depends on which code you used to arrive at your observation in the first place. Which app did you use?
I did not find a mention of the app ID or a link to your program code.

https://github.com/Viir/bots/blob/main/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

[Viir]

At the moment, your link points to https://github.com/Viir/bots/blob/0ef5dd76a8d116fe5c87bbb76a3a5996e69d04bf/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

It seems the image that you posted points to this part:

bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

Lines 621 to 628 in 0ef5dd7

	lockTargetFromOverviewEntry : OverviewWindowEntry -> BotDecisionContext -> DecisionPathNode
	lockTargetFromOverviewEntry overviewEntry context =
	describeBranch ("Lock target from overview entry '" ++ (overviewEntry.objectName |> Maybe.withDefault "") ++ "'")
	(useContextMenuCascadeOnOverviewEntry
	(useMenuEntryWithTextEqual "Lock target" menuCascadeCompleted)
	overviewEntry
	context
	)

Here we see the overview entry already comes in as an argument to this function. So to find the selection, we follow the references backward to where this function is applied.

I only found one application here:

bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

Lines 593 to 603 in 0ef5dd7

	lockTargetFromOverviewEntryAndEnsureIsInRange : BotDecisionContext -> Int -> OverviewWindowEntry -> DecisionPathNode
	lockTargetFromOverviewEntryAndEnsureIsInRange context rangeInMeters overviewEntry =
	case overviewEntry.objectDistanceInMeters of
	Ok distanceInMeters ->
	if distanceInMeters <= rangeInMeters then
	if overviewEntry.commonIndications.targetedByMe || overviewEntry.commonIndications.targeting then
	describeBranch "Locking target is in progress, wait for completion." waitForProgressInGame
	else
	describeBranch "Object is in range. Lock target."
	(lockTargetFromOverviewEntry overviewEntry context)

In this function, we also find a function argument that gives the overview entry.
So repeat going backward to applications on this function. This brings us here:

bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

Lines 491 to 507 in 0ef5dd7

	travelToMiningSiteAndLaunchDronesAndTargetAsteroid : BotDecisionContext -> DecisionPathNode
	travelToMiningSiteAndLaunchDronesAndTargetAsteroid context =
	case context.readingFromGameClient |> topmostAsteroidFromOverviewWindow of
	Nothing ->
	describeBranch "I see no asteroid in the overview. Warp to mining site."
	(returnDronesToBay context
	|> Maybe.withDefault
	(warpToMiningSite context)
	)
	Just asteroidInOverview ->
	describeBranch ("Choosing asteroid '" ++ (asteroidInOverview.objectName |> Maybe.withDefault "Nothing") ++ "'")
	(warpToOverviewEntryIfFarEnough context asteroidInOverview
	|> Maybe.withDefault
	(launchDrones context
	|> Maybe.withDefault
	(lockTargetFromOverviewEntryAndEnsureIsInRange

[zoom-maestro]

Just based on the title description of the function names it appears that this is where the magic is happening:

(Excuse me, I don't know how to reference the code like you do)

Where is the decision making to determine if "overviewEntry.commonIndications.targetedByMe || overviewEntry.commonIndications.targeting" is true or false?

[zoom-maestro]

@Viir I feel like I'm getting closer:

bots/implement/applications/eve-online/eve-online-mining-bot/EveOnline/ParseUserInterface.elm

Lines 1228 to 1334 in 0ef5dd7

	parseOverviewWindowEntry : List ( String, UITreeNodeWithDisplayRegion ) -> UITreeNodeWithDisplayRegion -> OverviewWindowEntry
	parseOverviewWindowEntry entriesHeaders overviewEntryNode =
	let
	textsLeftToRight =
	overviewEntryNode
	|> getAllContainedDisplayTextsWithRegion
	|> List.sortBy (Tuple.second >> .totalDisplayRegion >> .x)
	|> List.map Tuple.first
	cellsTexts =
	overviewEntryNode
	|> getAllContainedDisplayTextsWithRegion
	|> List.filterMap
	(\( cellText, cell ) ->
	let
	cellMiddle =
	cell.totalDisplayRegion.x + (cell.totalDisplayRegion.width // 2)
	maybeHeader =
	entriesHeaders
	|> List.filter
	(\( _, header ) ->
	header.totalDisplayRegion.x
	< cellMiddle
	+ 1
	&& cellMiddle
	< header.totalDisplayRegion.x
	+ header.totalDisplayRegion.width
	- 1
	)
	|> List.head
	in
	maybeHeader
	|> Maybe.map (\( headerText, _ ) -> ( headerText, cellText ))
	)
	|> Dict.fromList
	objectDistance =
	cellsTexts
	|> Dict.get "Distance"
	objectDistanceInMeters =
	objectDistance
	|> Maybe.map parseOverviewEntryDistanceInMetersFromText
	|> Maybe.withDefault (Err "Did not find the 'Distance' cell text.")
	spaceObjectIconNode =
	overviewEntryNode
	|> listDescendantsWithDisplayRegion
	|> List.filter (.uiNode >> .pythonObjectTypeName >> (==) "SpaceObjectIcon")
	|> List.head
	iconSpriteColorPercent =
	spaceObjectIconNode
	|> Maybe.map listDescendantsWithDisplayRegion
	|> Maybe.withDefault []
	|> List.filter (.uiNode >> getNameFromDictEntries >> (==) (Just "iconSprite"))
	|> List.head
	|> Maybe.andThen (.uiNode >> getColorPercentFromDictEntries)
	namesUnderSpaceObjectIcon =
	spaceObjectIconNode
	|> Maybe.map (.uiNode >> EveOnline.MemoryReading.listDescendantsInUITreeNode)
	|> Maybe.withDefault []
	|> List.filterMap getNameFromDictEntries
	|> Set.fromList
	bgColorFillsPercent =
	overviewEntryNode
	|> listDescendantsWithDisplayRegion
	|> List.filter (.uiNode >> .pythonObjectTypeName >> (==) "Fill")
	|> List.filter (.uiNode >> getNameFromDictEntries >> Maybe.map ((==) "bgColor") >> Maybe.withDefault False)
	|> List.filterMap (\fillUiNode -> fillUiNode.uiNode |> getColorPercentFromDictEntries)
	rightAlignedIconsHints =
	overviewEntryNode
	|> listDescendantsWithDisplayRegion
	|> List.filter (.uiNode >> getNameFromDictEntries >> Maybe.map ((==) "rightAlignedIconContainer") >> Maybe.withDefault False)
	|> List.concatMap listDescendantsWithDisplayRegion
	|> List.filterMap (.uiNode >> getHintTextFromDictEntries)
	rightAlignedIconsHintsContainsTextIgnoringCase textToSearch =
	rightAlignedIconsHints |> List.any (String.toLower >> String.contains (textToSearch |> String.toLower))
	commonIndications =
	{ targeting = namesUnderSpaceObjectIcon |> Set.member "targeting"
	, targetedByMe = namesUnderSpaceObjectIcon |> Set.member "targetedByMeIndicator"
	, isJammingMe = rightAlignedIconsHintsContainsTextIgnoringCase "is jamming me"
	, isWarpDisruptingMe = rightAlignedIconsHintsContainsTextIgnoringCase "is warp disrupting me"
	}
	in
	{ uiNode = overviewEntryNode
	, textsLeftToRight = textsLeftToRight
	, cellsTexts = cellsTexts
	, objectDistance = objectDistance
	, objectDistanceInMeters = objectDistanceInMeters
	, objectName = cellsTexts |> Dict.get "Name"
	, objectType = cellsTexts |> Dict.get "Type"
	, objectAlliance = cellsTexts |> Dict.get "Alliance"
	, iconSpriteColorPercent = iconSpriteColorPercent
	, namesUnderSpaceObjectIcon = namesUnderSpaceObjectIcon
	, bgColorFillsPercent = bgColorFillsPercent
	, rightAlignedIconsHints = rightAlignedIconsHints
	, commonIndications = commonIndications
	}

After doing a search on the word "parse", I also found this reference guide to the functions being used:
https://github.com/Viir/bots/blob/main/guide/eve-online/parsed-user-interface-of-the-eve-online-game-client.md

So, now that I've found WHERE the Overview items are being parsed down to their individual variables. I also see variables for an line item's name; type; and the iconSpriteColorPercent

I have still not yet discovered the script line that determines the selection of a valid target and/or the dismissal of an invalid target. I'll keep hunting.

[zoom-maestro]

I'm going to now read and study this guide:
https://github.com/Viir/bots/blob/main/guide/eve-online/developing-for-eve-online.md

I believe it should help me better understand how everything links together and communicate with you more clearly.

[Viir]

I'm going to now read and study this guide:
https://github.com/Viir/bots/blob/main/guide/eve-online/developing-for-eve-online.md

Thanks, let me know if anything is missing there.

I have still not yet discovered the script line that determines the selection of a valid target and/or the dismissal of an invalid target.

To get a better feel for how the app selects the next target, you can do a quick experiment. This takes less than five minutes:

• Change this function to return a different list item:

  bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

  Lines 1099 to 1103 in 0ef5dd7

  	topmostAsteroidFromOverviewWindow : ReadingFromGameClient -> Maybe OverviewWindowEntry
  	topmostAsteroidFromOverviewWindow =
  	overviewWindowEntriesRepresentingAsteroids
  	>> List.sortBy (.uiNode >> .totalDisplayRegion >> .y)
  	>> List.head

For example, use this instead:

topmostAsteroidFromOverviewWindow =
    overviewWindowEntriesRepresentingAsteroids
        >> List.sortBy (.uiNode >> .totalDisplayRegion >> .y)
        >> List.drop 1
        >> List.head

• Run a simulation with your new app code, replaying the same session where you made your earlier observation. (A simulation is faster than a live test, and you don't need to open a game client.)
• Compare the two instances of the event in which you saw the app click the overview entry. Here you can see the difference in behavior between the two app codes. So this illustrates the effect of a change in the app code in the game.

[zoom-maestro]

I'm going to now read and study this guide:
https://github.com/Viir/bots/blob/main/guide/eve-online/developing-for-eve-online.md

Thanks, let me know if anything is missing there.

Will do.

I have still not yet discovered the script line that determines the selection of a valid target and/or the dismissal of an invalid target.

To get a better feel for how the app selects the next target, you can do a quick experiment. This takes less than five minutes:

• Change this function to return a different list item:

  bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

  Lines 1099 to 1103 in 0ef5dd7

  	topmostAsteroidFromOverviewWindow : ReadingFromGameClient -> Maybe OverviewWindowEntry
  	topmostAsteroidFromOverviewWindow =
  	overviewWindowEntriesRepresentingAsteroids
  	>> List.sortBy (.uiNode >> .totalDisplayRegion >> .y)
  	>> List.head

For example, use this instead:

topmostAsteroidFromOverviewWindow =
    overviewWindowEntriesRepresentingAsteroids
        >> List.sortBy (.uiNode >> .totalDisplayRegion >> .y)
        >> List.drop 1
        >> List.head

Wow, I'm an ID-10-T. I guess at some point while looking at BotEngineApp.elm I decided to stop scrolling down. This is exactly what I was looking for. Even more specific is THIS:

bots/implement/applications/eve-online/eve-online-mining-bot/BotEngineApp.elm

Lines 1113 to 1117 in 0ef5dd7

	overviewWindowEntryRepresentsAnAsteroid : OverviewWindowEntry -> Bool
	overviewWindowEntryRepresentsAnAsteroid entry =
	(entry.textsLeftToRight |> List.any (String.toLower >> String.contains "asteroid"))
	&& (entry.textsLeftToRight |> List.any (String.toLower >> String.contains "belt") |> not)

I understand this block as the criteria used to MAKE the actual list. This is what I was looking for. It creates a list based on the criteria that any row of the Overview shall contain the string "asteroid" in it BUT not contain the string "belt". Going into the Parser:

bots/implement/applications/eve-online/eve-online-mining-bot/EveOnline/ParseUserInterface.elm

Lines 237 to 252 in 0ef5dd7

	type alias OverviewWindowEntry =
	{ uiNode : UITreeNodeWithDisplayRegion
	, textsLeftToRight : List String
	, cellsTexts : Dict.Dict String String
	, objectDistance : Maybe String
	, objectDistanceInMeters : Result String Int
	, objectName : Maybe String
	, objectType : Maybe String
	, objectAlliance : Maybe String
	, iconSpriteColorPercent : Maybe ColorComponents
	, namesUnderSpaceObjectIcon : Set.Set String
	, bgColorFillsPercent : List ColorComponents
	, rightAlignedIconsHints : List String
	, commonIndications : OverviewWindowEntryCommonIndications
	}

I see textsLeftToRight as a attribute variable. I also see iconSpriteColorPercent and bgColorFillsPercent . I think this could be added as extra logic criteria to narrow down the selection options for topmostAsteroidFromOverWindow.

maybe something like:

overviewWindowEntryRepresentsAnAsteroid : OverviewWindowEntry -> Bool
overviewWindowEntryRepresentsAnAsteroid entry =
    (entry.textsLeftToRight |> List.any (String.toLower >> String.contains "asteroid"))
        && (entry.textsLeftToRight |> List.any (String.toLower >> String.contains "belt") |> not)
        && (entry.iconSpriteColorPercent |> List.any(????) String.contains "???" |> not)

Just need to figure out the valid values of the Overview entries for this:

and whether or not iconSpriteColorPercent and bgColorFillsPercent are relevant to that shaded entry.

[Viir]

and whether or not iconSpriteColorPercent and bgColorFillsPercent are relevant to that shaded entry.

If the values in these properties are not sufficient to distinguish the entries, the next step could be checking what values your game client provided in uiNode.
The properties you mentioned only contain some derivation of the content in uiNode. Maybe you are the first to encounter the shaded entry problem, so there might be no matching derivation in the parse module so far.
You can use the tree view in the alternate UI for EVE Online to explore the values there.
You don't have to run the full alternate UI in this case. You can instead load a reading from a file with this precompiled page: https://botengine.blob.core.windows.net/blob-library/by-name/2020-12-11-eve-online-alternate-ui.html

This walkthrough shows the process to inspect a reading from the EVE Online game client: https://vimeo.com/user132945801/making-an-eve-online-bot-see-anomalies-and-other-pilots#t=240s

One more thing I noticed when looking at your screenshot: bgColorFillsPercent depends on selection so that it could contain a misleading difference.

[ls400hurdles]

AH! much more detailed lol This is what i was referencing in my last comment :D Has this been solved?

[Viir]

@ls400hurdles Yes, if you found that iconSpriteColorPercent is predicting this for your setup, add a function like the following:

overviewWindowEntryLooksValid : OverviewWindowEntry -> Bool
overviewWindowEntryLooksValid entry =
    entry.iconSpriteColorPercent
        == Just iconSpriteColorPercent_predicting_valid_overview_entry__on_setup_from_ls400hurdles


iconSpriteColorPercent_predicting_valid_overview_entry__on_setup_from_ls400hurdles : EveOnline.ParseUserInterface.ColorComponents
iconSpriteColorPercent_predicting_valid_overview_entry__on_setup_from_ls400hurdles =
    { a = 1234, r = 1234, g = 1234, b = 1234 }

Then integrate it into overviewWindowEntryRepresentsAnAsteroid as follows:

overviewWindowEntryRepresentsAnAsteroid : OverviewWindowEntry -> Bool
overviewWindowEntryRepresentsAnAsteroid entry =
    (entry.textsLeftToRight |> List.any (stringContainsIgnoringCase "asteroid"))
        && (entry.textsLeftToRight |> List.any (stringContainsIgnoringCase "belt") |> not)
        && overviewWindowEntryLooksValid entry

You can find the newest version of the memory reading inspection tool at https://botlabs.blob.core.windows.net/blob-library/by-name/2022-05-01-eve-online-alternate-ui.html

The guide on the parsed memory reading is now at https://to.botlab.org/guide/parsed-user-interface-of-the-eve-online-game-client

[HongLouWang]

Want this future be deployed as soon