[WotLK] MoveMapGen.sh - For 8 processors, and improve map process distribution

[adellaci]

I have been worker on changes for the MoveMapGen.sh so that I could make better use of the core that I have in my system. I have set this one up to use 1,2,4, or 8 processes. The change has no impact to 1 process.

While working on this I discovered that maps 605, and 606 were not being processed. These to maps show 9 and 2 entries respectively in vmaps.

Also found the distribution of maps(tiles) to be processed were not even. The changes here help even the number of tiles being processed across the processes. Testing with a 8 I show noticeable reduction in time to process all the files.

Could some one please confirm

MoveMapGen8Core.sh

#!/bin/sh

# This file is part of the CMaNGOS Project. See AUTHORS file for Copyright information
#
# This file is free software; as a special exception the author gives
# unlimited permission to copy and/or distribute it, with or without
# modifications, as long as this notice is preserved.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
# implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

## Syntax of this helper
## First param must be number of to be used CPUs (only 1, 2, 4, 8 supported) or "offmesh" to recreate the special tiles from the OFFMESH_FILE
## Second param can be an additional filename for storing log
## Third param can be an addition filename for storing detailed log

## Additional Parameters to be forwarded to MoveMapGen, see mmaps/readme for instructions
PARAMS="--silent"

## Already a few map extracted, and don't care anymore
EXCLUDE_MAPS=""
#EXCLUDE_MAPS="0 1 530 571" # example to exclude the continents

## Offmesh file
OFFMESH_FILE="offmesh.txt"

## Normal log file (if not overwritten by second param
LOG_FILE="MoveMapGen.log"
## Detailed log file
DETAIL_LOG_FILE="MoveMapGen_detailed.log"

## ! Use below only for finetuning or if you know what you are doing !

## All maps
MAP_LIST_A="571"
MAP_LIST_B="1"
MAP_LIST_C="530 44 550 552 553 554 555 556 557 558"
MAP_LIST_D="0"
MAP_LIST_E="599 600 608 616 649 723 573 37 534 564 580 349 369 389 409 429 449 450 451 565 576 582 584 586 587 588 589 13 25 34 35 42 43"
MAP_LIST_F="47 632 532 559 562 617 618 624 658 289 469 489 529 601 615 650 724 329 668 209 129 533 604 33 309 531 568 572 540 542 544 545 546 547 548 578 48 70 90 109 229 230 249 590 591 592 593 594 596 598"
MAP_LIST_G="30 36 189 560 566 595 269 509 609 602 619 574 575"
MAP_LIST_H="607 543 603 628 631 585 169"

badParam()
{
  echo "ERROR! Bad arguments!"
  echo "You can (re)extract mmaps with this helper script,"
  echo "or recreate only the tiles from the offmash file"
  echo
  echo "Call with number of processes (1,2,4,8) to create mmaps"
  echo "Call with 'offmesh' to reextract the tiles from offmash file"
  echo "Example: ./MoveMapGen.sh 8"
  echo
  echo "For further fine-tuning edit this helper script"
  echo
}

if [ "$#" = "3" ]
then
  LOG_FILE=$2
  DETAIL_LOG_FILE=$3
elif [ "$#" = "2" ]
then
  LOG_FILE=$2
fi

# Offmesh file provided?
OFFMESH=""
if [ "$OFFMESH_FILE" != "" ]
then
  if [ ! -f "$OFFMESH_FILE" ]
  then
    echo "ERROR! Offmesh file $OFFMESH_FILE could not be found."
    echo "Provide valid file or none. You need to edit the script"
    exit 1
  else
    OFFMESH="--offMeshInput $OFFMESH_FILE"
  fi
fi

# Function to process a list
createMMaps()
{
  for i in $@
  do
    for j in $EXCLUDE_MAPS
    do
      if [ "$i" = "$j" ]
      then
        continue 2
      fi
    done
    ./MoveMapGen $PARAMS $OFFMESH $i | tee -a $DETAIL_LOG_FILE
    echo "`date`: (Re)created map $i" | tee -a $LOG_FILE
  done
}

createHeader()
{
  echo "`date`: Start creating MoveMaps" | tee -a $LOG_FILE
  echo "Used params: $PARAMS $OFFMESH" | tee -a $LOG_FILE
  echo "Detailed log can be found in $DETAIL_LOG_FILE" | tee -a $LOG_FILE
  echo "Start creating MoveMaps" | tee -a $DETAIL_LOG_FILE
  echo
  echo "Be PATIENT - This will take a long time and might also have gaps between visible changes on the console."
  echo "WAIT until you are informed that 'creating MoveMaps' is 'finished'!"
}

# Create mmaps directory if not exist
if [ ! -d mmaps ]
then
  mkdir mmaps
fi

# Param control
case "$1" in
  "1" )
    createHeader $1
    createMMaps $MAP_LIST_A $MAP_LIST_B $MAP_LIST_C $MAP_LIST_D $MAP_LIST_E $MAP_LIST_F $MAP_LIST_G $MAP_LIST_H &
    ;;
  "2" )
    createHeader $1
    createMMaps $MAP_LIST_A $MAP_LIST_D $MAP_LIST_F $MAP_LIST_H &
    createMMaps $MAP_LIST_B $MAP_LIST_C $MAP_LIST_E $MAP_LIST_G &
    ;;
  "4" )
    createHeader $1
    createMMaps $MAP_LIST_A $MAP_LIST_E &
    createMMaps $MAP_LIST_B $MAP_LIST_F &
    createMMaps $MAP_LIST_C $MAP_LIST_G &
    createMMaps $MAP_LIST_D $MAP_LIST_H &
    ;;
  "8" )
    createHeader $1
    createMMaps $MAP_LIST_A &
    createMMaps $MAP_LIST_E &
    createMMaps $MAP_LIST_B &
    createMMaps $MAP_LIST_F &
    createMMaps $MAP_LIST_C &
    createMMaps $MAP_LIST_G &
    createMMaps $MAP_LIST_D &
    createMMaps $MAP_LIST_H &
    ;;
  "offmesh" )
    echo "`date`: Recreate offmeshs from file $OFFMESH_FILE" | tee -a $LOG_FILE
    echo "Recreate offmeshs from file $OFFMESH_FILE" | tee -a $DETAIL_LOG_FILE
    while read map tile line
    do
      ./MoveMapGen $PARAMS $OFFMESH $map --tile $tile | tee -a $DETAIL_LOG_FILE
      echo "`date`: Recreated $map $tile from $OFFMESH_FILE" | tee -a $LOG_FILE
    done < $OFFMESH_FILE &
    ;;
  * )
    badParam
    exit 1
    ;;
esac

wait

echo  | tee -a $LOG_FILE
echo  | tee -a $DETAIL_LOG_FILE
echo "`date`: Finished creating MoveMaps" | tee -a $LOG_FILE
echo "`date`: Finished creating MoveMaps" >> $DETAIL_LOG_FILE

[evil-at-wow]

While working on this I discovered that maps 605, and 606 were not being processed.

I believe that was done on purpose, because these maps are not actually used ingame (map 605 = Development Land (non-weighted textures) and map 606 = QA and DVD).

[adellaci]

Thanks for the info. I'll pull those two out

[Muehe]

Just tested this using 8 cores, works fine. 👍

Overall time:
03:43:16 - 04:19:44 (00:36:28)

Map groups finish time:

1. e 03:57:56
2. f 04:04:41
3. g 04:08:48
4. h 04:10:45
5. d 04:12:13
6. c 04:12:59
7. a 04:14:06
8. b 04:19:44

[adellaci]

Thank you for posting the results

[adellaci]

Made another adjustment to the distribution of map processing. Map 1 is the longest to process and is the factor that determines how long.

[adellaci]

I going to make this comment for WotLK MoveMapGen.sh Sunday afternoon, if no one has any comments, suggestions, or objections.
[https://github.com/adellaci/mangos-wotlk/pull/1]

[adellaci]

Commit Completed