OSRS Integration #473

Khaled431 · 2020-02-29T21:11:14Z

Start of the potential move to OSRS.

garyttierney · 2020-02-29T21:13:05Z

Related to #472.

…t.xml file has duplicates in it.

Renamed XteaDecoder.java to XteaParser.java Added XteaUtil which decrypts/encrypts a ByteBuf with Xtea rounds. Added UserStats which details the incoming connection's hardware specs.

Added user stats to the credentials to replace uid.

Resolved issue with Crc check being invalid due to index sixteen being manually zeroed over the wire by Jagex.

… someone decides how to move forward with a multi-world impl.

Added the following decode messages: Add/Remove Ignores&Friends, Private Chat Messages.

Report abuse decoder. Keepalive decoder. Walk decoder. Arrowkey decoder. Close interface decoder. Amount input decoder. Removed 317 rev and 377 to follow soon. Note: You may need to exclude the 377 directory manually.

Object options decoders.

… organized but if anyone has suggestions I will gladly listen.

…ws-1252 characters. More decoding messages, have these remaining so far: ButtonMessageDecoder DialogueContinueMessageDecoder FlaggedMouseEventMessageDecoder ItemOnItemMessageDecoder ItemOnNpcMessageDecoder ItemOnObjectMessageDecoder SwitchItemMessageDecoder TakeTileItemMessageDecoder MagicOnItemMessageDecoder MagicOnNpcMessageDecoder MagicOnPlayerMessageDecoder FirstItemOptionMessageDecoder SecondItemOptionMessageDecoder ThirdItemOptionMessageDecoder FourthItemOptionMessageDecoder FifthItemOptionMessageDecoder PlayerDesignMessageDecoder PublicChatMessageDecoder Doesn't exist anymore: FlashingTabClickedMessageDecoder FirstItemActionMessageDecoder SecondItemActionMessageDecoder ThirdItemActionMessageDecoder FourthItemActionMessageDecoder FifthItemActionMessageDecoder

NPCS are complete w/o masks.

EnumDefinition.java support for OSRS Enums. DisplayStatusMessage has been supported (The ability to switch the display mode).

Standard named for id << component should be packedInterface. Added IfSetEventMessage encoder.

Added KeepAliveMessageDecoder message.

Directions have client rotations now.

PublicChatDecoder updated. Public chat is now working.

Fixes issue where we send amount when item is being removed in UpdateContainerPartialMessageEncoder

Moved equipment handler to an IfActionMessage

…andle 3000.

Half of social stuff done, adding friends communication between them.

…private chat.

garyttierney

This is a first pass to get rid of the code churn in this PR so I can review the actual code changes. We need to get rid of the commit that deletes the Ruby and the commits that optimize imports and I think this will be easier to review.

garyttierney · 2020-04-25T22:12:00Z

cache/build.gradle

@@ -5,6 +5,10 @@ description = 'Apollo Cache'
 dependencies {
    implementation project(':util')
    implementation group: 'com.google.guava', name: 'guava', version: guavaVersion
+    implementation group: 'org.tukaani', name: 'xz', version: '1.8'
+    implementation group: 'it.unimi.dsi', name: 'fastutil', version: '8.3.1'
+    implementation group: 'org.apache.ant', name: 'ant', version: '1.10.7'


Why are we depending on ant?

So... can it be removed?

garyttierney · 2020-04-25T22:14:12Z

cache/build.gradle

+    implementation group: 'org.tukaani', name: 'xz', version: '1.8'
+    implementation group: 'it.unimi.dsi', name: 'fastutil', version: '8.3.1'
+    implementation group: 'org.apache.ant', name: 'ant', version: '1.10.7'
+    implementation group: 'com.google.code.gson', name: 'gson', version: '2.8.6'


I'd recommend using Jackson instead of Gson.

Sure, no difference to me.

garyttierney · 2020-04-25T22:16:10Z

cache/src/main/java/org/apollo/cache/decoder/NpcDefinitionDecoder.java

-
-import org.apollo.cache.IndexedFileSystem;
-import org.apollo.cache.archive.Archive;
+import it.unimi.dsi.fastutil.ints.Int2ObjectArrayMap;


Use standard collections instead of these fastutil collections. We optimize on a case by case basis, not generally like this.

That's fine. I'll get rid of it.

garyttierney · 2020-04-25T22:17:20Z

cache/src/main/java/org/apollo/cache/decoder/ObjectDefinitionDecoder.java

+				definition.setF2117(buffer.readUShort());
+				definition.setF2118(buffer.readUByte());
+			} else if (opcode == 79) {
+				definition.setF2119(buffer.readUShort());


If a field is unknown we don't assign it to unused fields, instead we just skip the data.

To be faaaaaaaiiiiiiir I c&p this out of my cache editor. But i'll get rid of them.

garyttierney · 2020-04-25T22:18:29Z

game/data/messages.xml

@@ -1,26 +1,41 @@
 <messages>
    <message>
-        <type>org.apollo.game.message.impl.ButtonMessage</type>
+        <type>org.apollo.game.message.impl.decode.IfActionMessage</type>


These class names aren't descriptive. They need to have meaningful names without abbreviations.

That's the actual name Jagex use., I prefer it but I can see how it's not descriptive. I'll change it back.

That's the actual name Jagex use

This is a fairly tired argument. A lot of what Jagex done is garbage.

I'm not disagreeing, I'm just explaining the decision.

garyttierney · 2020-04-25T22:20:09Z

game/src/main/java/org/apollo/game/fs/decoder/SynchronousDecoder.java

-import java.util.concurrent.Executors;
-import java.util.concurrent.Future;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.*;


You've ran IntelliJ's "optimize imports" function over the code, this isn't the place to be doing that.

I have it set to auto do it. Sorry.

garyttierney · 2020-04-25T22:20:56Z

game/src/main/java/org/apollo/game/message/handler/BankMessageHandler.java

-		if (player.getInterfaceSet().contains(BankConstants.BANK_WINDOW_ID)) {
-			if (message.getInterfaceId() == BankConstants.SIDEBAR_INVENTORY_ID) {
+		if (player.getInterfaceSet().contains(BankConstants.WINDOW_ID)) {
+			//if (message.getInterfaceId() == BankConstants.SIDEBAR_INVENTORY_ID) {


Odd code here. Why is this commented out?

Honestly I don't remember. All interfaces have to be redone regardless.

@garyttierney It's because the interface doesn't exist anymore. I've removed it entirely now.

garyttierney · 2020-04-25T22:55:41Z

net/src/main/java/org/apollo/net/update/resource/WorldListResourceProvider.java

-		world.writeByte(0); // country
-		world.writeShort(2000);// number of players
-		return world;
+	private ByteBuffer createWorld() {


Do we actually need this to get login working?

Client freezes without it when clicked on by mistake.

Hm, then I'd be inclined to make it return real data. cc @Major-

garyttierney · 2020-04-26T13:20:21Z

game/src/main/java/org/apollo/game/message/impl/decode/OpItemMessage.java

+/**
+ * The type Op item message.
+ */
+public class OpItemMessage extends Message {


Naming needs reverted.

Reverted to what exactly? Apollo never had ground items.

Then pick something sensible that isn't nonsense like OpItem.

game/src/main/java/org/apollo/game/model/area/update/UpdateOperation.java

game/src/main/java/org/apollo/game/release/r181/decoders/EventKeyboardDecoder.java

garyttierney · 2020-05-03T14:33:03Z

util/src/main/java/org/apollo/util/security/UserStats.java

+	 *
+	 * @param buffer the buffer
+	 */
+	public UserStats(ByteBuf buffer) {


A constructor isn't the place to be reading from a byte buffer. Create a static constructor and make it clear that the buffer is consumed.

net/src/main/java/org/apollo/net/codec/update/OnDemandResponseEncoder.java

util/src/main/java/org/apollo/util/BufferUtil.java

game/src/main/java/org/apollo/game/release/r181/decoders/EventKeyboardDecoder.java

garyttierney · 2020-05-03T15:01:28Z

game/src/main/java/org/apollo/game/model/entity/Player.java

@@ -724,7 +735,8 @@ public void send(Message message) {
 	 * Sends the initial messages.
 	 */
 	public void sendInitialMessages(DisplayMode mode) {
-		send(new RebuildNormalMessage(position, index, world.getRegionRepository().getXteaRepository(), false));
+		send(new RebuildNormalMessage(position, index, updateInfo, world.getPlayerRepository(),


This is an odd way to pass data for initialization, cc @Major-, we need to look at setting up player "pre-init" code.

I wasn't sure how to pass it around. You could do everything with a single player reference but I tried to keep it consistent with how other things are done.

garyttierney · 2020-05-03T15:04:31Z

game/src/main/java/org/apollo/game/sync/task/PlayerSynchronizationTask.java

 	 */
-	private static final int MAXIMUM_LOCAL_PLAYERS = 255;
+	private void prefetch() {
+		limitedPlayers.clear();


It looks like this is an approximation of:

for (Player updatingPlayer : playersToUpdate) { for (Region region : regions) { for (Player candidate: regions.getPlayers()) { // test } } }

is that right?

Hm, this is what we do currently. That doesn't quite scale. @Major- @rmcmk ever thought about this? Scanning close regions or adding to this change buffer with an event system seems like a better idea.

That's not exactly correct.

for (Region region : regions) {
for (Player candidate: regions.getPlayers()) {
// test
}
}

Is more what it is like in that function. If you're referring to what it does for every player combined,, then yes.

Major- · 2020-05-03T20:50:30Z

cache/build.gradle

@@ -5,6 +5,10 @@ description = 'Apollo Cache'
 dependencies {
    implementation project(':util')
    implementation group: 'com.google.guava', name: 'guava', version: guavaVersion
+    implementation group: 'org.tukaani', name: 'xz', version: '1.8'
+    implementation group: 'it.unimi.dsi', name: 'fastutil', version: '8.3.1'
+    implementation group: 'org.apache.ant', name: 'ant', version: '1.10.7'


So... can it be removed?

cache/src/main/java/org/apollo/cache/Archive.java

cache/src/main/java/org/apollo/cache/CacheBuffer.java

cache/src/main/java/org/apollo/cache/Helper.java

softagram · 2020-05-13T00:01:09Z

Softagram Impact Report for pull/473 (head commit: `ba35a57`)

⚠️ Copy paste found (largest 3 shown)

ℹ️ ItemOnObjectVerificationHandlerTests.java: Copy paste fragment inside the same file on lines 47, 79:


  World world = mock(World.class);
  Region region = mock(Region.class);
  RegionRepository regionRepository = mock(RegionRepository.class);
  Pl...(truncated 788 chars)

ℹ️ ItemOnObjectVerificationHandlerTests.java: Copy paste fragment inside the same file on lines 56, 121:


  when(player.getInventory()).thenReturn(inventory);
  when(world.getRegionRepository()).thenReturn(regionRepository);
  when(regionRepository.fromPosition(...(truncated 733 chars)

ℹ️ ObjectActionVerificationHandlerTests.java: Copy paste fragment inside the same file on lines 49, 75:


  when(world.getRegionRepository()).thenReturn(regionRepository);
  when(regionRepository.fromPosition(objectPosition)).thenR...(truncated 598 chars)

Now that you are on the file, it would be easier to pay back some tech. debt.

⭐ Change Overview

(Open in Softagram Desktop for full details)

⭐ Details of Dependency Changes

(Open in Softagram Desktop for full details)

📄 Full report

Permalink: Full report for pull/473

Impact Report explained. Give feedback on this report to support@softagram.com

Cache definitions, update server and most of login done for rev 181.

0930e66

Khaled Jaber added 6 commits February 29, 2020 16:13

Replaced numeric constant '181' with the release revision.

910a9c6

Bumped the JDK version in the azure-pipelines.yml to 11 from 1.8.

36ee4c0

Binding on the same port is now filtered out. This occurs when the ne…

1f6041c

…t.xml file has duplicates in it.

Login decoding updated to OSRS.

662570e

Renamed XteaDecoder.java to XteaParser.java Added XteaUtil which decrypts/encrypts a ByteBuf with Xtea rounds. Added UserStats which details the incoming connection's hardware specs.

Documented user stats.

b901cb5

Added user stats to the credentials to replace uid.

Updated UserStats to revision 8.

0cf9710

Resolved issue with Crc check being invalid due to index sixteen being manually zeroed over the wire by Jagex.

Khaled431 changed the title ~~Cache definitions, update server and most of login done for rev 181.~~ OSRS Integration Mar 1, 2020

Major- added the osrs label Mar 1, 2020

Khaled Jaber added 20 commits March 1, 2020 18:08

Worldlist is now available. Currently it's just magic constants until…

86cc048

… someone decides how to move forward with a multi-world impl.

Worldlist is now available. Currently it's just magic constants until…

72292a2

… someone decides how to move forward with a multi-world impl.

Added the following encode messages: Varp

071b04d

Added the following decode messages: Add/Remove Ignores&Friends, Private Chat Messages.

Player options decoders.

570745a

Report abuse decoder. Keepalive decoder. Walk decoder. Arrowkey decoder. Close interface decoder. Amount input decoder. Removed 317 rev and 377 to follow soon. Note: You may need to exclude the 377 directory manually.

Npc options decoders.

a45b4d3

Object options decoders.

Refractored the directories in which each file lives. This seems more…

e0e3bd7

… organized but if anyone has suggestions I will gladly listen.

Refractored the directories in which each file lives. This seems more…

b0c0918

… organized but if anyone has suggestions I will gladly listen.

Login to World complete.

b4a3a62

NPCS are complete w/o masks.

InterfaceSet.java is updated to support osrs functionality.

3ef4250

EnumDefinition.java support for OSRS Enums. DisplayStatusMessage has been supported (The ability to switch the display mode).

Reduces the move-toplevel code.

4362b4f

Experience and Weight Encoder messages.

db9a443

Added the run-energy message.

ccc7e0f

Organized imports.

9fcb2fd

Added the plugins for shops back with proper npc ids.

3ac6b6b

Added the plugins for shops back with proper npc ids.

7fe0272

Fixes spawns with modern names.

f972fd1

Gets rid of leak from the ondemand response encoder.

a0d9869

Standard named for id << component should be packedInterface. Added IfSetEventMessage encoder.

Added SetPlayerActionMessageEncoder message.

060053e

Added KeepAliveMessageDecoder message.

Inventory messages are complete.

948fad1

Khaled Jaber added 20 commits April 12, 2020 11:20

Added tutorial island wood and oak tree identifiers.

65309ab

Directions have client rotations now.

Fixes title mask flag.

15833bf

Keyboard event added.

bbc0e13

ChatFilterSettingsMessage added.

ac00bc2

PublicChatDecoder updated. Public chat is now working.

Removed item option, we should have one consistent message for that.

c858ac4

Fixes issue with inventory and component id being swapped.

5037fff

Fixes issue where we send amount when item is being removed in UpdateContainerPartialMessageEncoder

Removed some debug statements from GPI.

88928e9

Moved equipment handler to an IfActionMessage

Zone packets, add and remove object messages.

deb1a17

Added proper priorities and the UpdateZonePartialEnclosedEncoder.

c9f5e73

Finished supported map zone packets.

0c92c9f

Added unsupported map zone packets.

a15165b

Fixes issue with crash when changing regions.

d56d375

Adds more packets to queue, previous limit was 25 packets. Osrs can h…

4cbe700

…andle 3000.

Adds huffman codec class to properly compress/decompress mesasges.

7b5b158

Half of social stuff done, adding friends communication between them.

Fixes problem that broke pub chat. Still need to fix compression for …

4af53e9

…private chat.

Fixes issue with HuffmanCodec

4bddfe2

Fixes codec issues.

04ab087

Fixes issue with HuffmanCodec

79ac1a8

Friends list interactions complete.

ba28a23

Ignores list complete.

e393743

garyttierney requested changes Apr 25, 2020

View reviewed changes

garyttierney reviewed Apr 25, 2020

View reviewed changes

All the changes that sfix has asked for.

bfb2619

garyttierney requested changes May 3, 2020

View reviewed changes

garyttierney reviewed May 3, 2020

View reviewed changes

Major- reviewed May 3, 2020

View reviewed changes

Code review pass apollo-rsps#2 osrs-proposal

ba35a57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OSRS Integration #473

OSRS Integration #473

Khaled431 commented Feb 29, 2020

garyttierney commented Feb 29, 2020

garyttierney left a comment

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

Major- May 3, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020 •

edited

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

Khaled431 Apr 26, 2020

garyttierney Apr 25, 2020

Khaled431 Apr 25, 2020

garyttierney Apr 25, 2020

garyttierney Apr 26, 2020

Khaled431 May 12, 2020

garyttierney May 13, 2020

garyttierney May 3, 2020

garyttierney May 3, 2020

Khaled431 May 3, 2020

garyttierney May 3, 2020

garyttierney May 3, 2020

Khaled431 May 3, 2020

Major- May 3, 2020

softagram bot commented May 13, 2020

OSRS Integration #473

Are you sure you want to change the base?

OSRS Integration #473

Conversation

Khaled431 commented Feb 29, 2020

garyttierney commented Feb 29, 2020

garyttierney left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Khaled431 Apr 25, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

softagram bot commented May 13, 2020

Softagram Impact Report for pull/473 (head commit: ba35a57)

⚠️ Copy paste found (largest 3 shown)

⭐ Change Overview

⭐ Details of Dependency Changes

📄 Full report

Khaled431 Apr 25, 2020 •

edited

Softagram Impact Report for pull/473 (head commit: `ba35a57`)