-
Notifications
You must be signed in to change notification settings - Fork 1
/
#neural-net.lyx#
executable file
·441 lines (351 loc) · 11.2 KB
/
#neural-net.lyx#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
#LyX 2.0 created this file. For more info see http://www.lyx.org/
\lyxformat 413
\begin_document
\begin_header
\textclass llncs
\begin_preamble
%\documentclass[a4paper,10pt]{scrartcl}
\title{Memory-Based Reasoning for Artificial Intelligence in Computer Games \\ {\small CS6250 - Course Project Proposal} }
\author{\it Sahitya Potluri, Vignesh Krishnakumar, Abhiram Ravi}
\institute{Indian Institute of Technology, Madras}
\pdfinfo{%
/Title ()
/Author ()
/Creator ()
/Producer ()
/Subject ()
/Keywords ()
}
\end_preamble
\use_default_options false
\maintain_unincluded_children false
\language english
\language_package none
\inputencoding utf8
\fontencoding default
\font_roman palatino
\font_sans cmss
\font_typewriter lmtt
\font_default_family sfdefault
\use_non_tex_fonts false
\font_sc false
\font_osf false
\font_sf_scale 100
\font_tt_scale 100
\graphics default
\default_output_format default
\output_sync 0
\bibtex_command default
\index_command default
\paperfontsize 12
\spacing single
\use_hyperref false
\papersize a4paper
\use_geometry true
\use_amsmath 1
\use_esint 1
\use_mhchem 0
\use_mathdots 0
\cite_engine basic
\use_bibtopic false
\use_indices false
\paperorientation portrait
\suppress_date true
\use_refstyle 0
\index Index
\shortcut idx
\color #008000
\end_index
\leftmargin 3cm
\topmargin 3.5cm
\rightmargin 3cm
\bottommargin 3.5cm
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\paragraph_indentation default
\quotes_language english
\papercolumns 1
\papersides 1
\paperpagestyle default
\tracking_changes false
\output_changes false
\html_math_output 0
\html_css_as_file 0
\html_be_strict false
\end_header
\begin_body
\begin_layout Title
Memory Based Reasoning in Artificial Intelligence for Racing Cars
\end_layout
\begin_layout Abstract
Computer games have been a playground for academicians in the experimentation
of several Artificial Intelligence techniques.
Game Bots (a.k.a Game AI) borrow several techniques from the broader field
of AI and Machine Learning in bringing human-level cognitive processes
to the non-player characters/bots in the game, providing a strong platform
for the simulation of AI techniques that are applicable to the real world.
In this course, we experiment with the application of Case-based reasoning
\begin_inset Note Note
status open
\begin_layout Plain Layout
and Fuzzy logic controllers
\end_layout
\end_inset
in the building of an AI-driven car for the Open Source Car-Racing game
\shape italic
Speed-dreams
\shape default
(a.k.a TORCS).
TORCS is a famous open source racing game project, which has been used
extensively by academicians for game AI research.
\end_layout
\begin_layout Section*
Introduction
\end_layout
\begin_layout Standard
Virtual Gaming Environments are killer apps for the application of Human-level
Artificial Intelligence.
Although AI techniques have been applied to games such as Chess, Checkers
etc., the traditional techniques fail in the domain of more complicated
games such as Multiplayer Role-playing games or real time strategy games
because the decision space is too vast for making decisions in real-time.
Traditional Machine learning techniques, where we predict an action based
on the state, also fail to perform, since we would require a dataset that
is exponential in the number of elements that describe a state in the game.
This introduces the need for an efficient memory model and additional reasoning
to adapt previously seen instances to a newly encountered scene.
We tackle the situation of a Racing-car scenario, where we would like an
AI to learn to drive by observing other expert cars play.
In our approach, the AI builds a case base by observing the actions of
an expert racer and uses this case base to decide actions given a new scenario.
The rest of this paper is organized as follows - We first present a detailed
outline of our approach.
We then present the details of implementation on the Speed-dreams simulator.
We then evaluate the performance of our approach by running our AI car
on an
\emph on
unseen
\emph default
racing track.
\end_layout
\begin_layout Section*
Case Based Reasoning for Racing Cars
\end_layout
\begin_layout Standard
The goal of our approach is to build an AI racing car that learns to play
by studying the actions of expert racers on various scenarios.
By exploiting the paradigm of case-based reasoning, our AI learns to conquer
the road with the following three steps
\end_layout
\begin_layout Enumerate
\series bold
Case Base Construction
\series default
- We first let expert racers run on a variety of different racing tracks,
and log information regarding the actions taken by these experts for different
state parameters.
The problem attributes include some crucial parameters involving the internal
and external states of the car along with parameters of the portion of
the track local to the position of the car.
The lesson attributes include the control parameters of the car i.e the
decisions takes by the expert racer during the corresponding problem scenario.
This is done by running a single expert car (denote
\series bold
E
\series default
) on a variety of training tracks (denote
\series bold
TT
\series default
).
For reasons of simplicity, we work with a single car model
\begin_inset Newline newline
\end_inset
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\align center
\begin_inset Graphics
filename /home/abhiram/Documents/academics/8/CS6250 - Memory Based Reasoning in AI/project/FCHART.png
width 70page%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption
\begin_layout Plain Layout
An overview of the case base, with the comprehensive list of attributes
(both problem and lesson)
\end_layout
\end_inset
\end_layout
\begin_layout Plain Layout
\end_layout
\end_inset
\end_layout
\begin_layout Enumerate
\series bold
Similarity Computation
\series default
- Since we would like to run our AI car on a previously unseen track, we
would like to approximate the newly encountered situation to seen data
from the case base, and obtain the
\emph on
nearest
\emph default
neighbors, before adapting it to make our decision.
Computing the nearest neighbors requires the description of a similarity
measure between two problems.
We experiment with several local and global similarity measures including
the default one provided by the myCBR framework.
The global similarity measure is a weighted average of all the local similarity
measures between the individual attributes.
The weights that we used were inferred from the importance of the physical
interpretation of the attributes in making the control decisions of the
car\SpecialChar \@.
\begin_inset Newline newline
\end_inset
\end_layout
\begin_layout Enumerate
\series bold
Solution Adaptation
\series default
- It is not practically feasible to directly use the solution of the nearest
neighbor of the problem in the case base, as the situation may be semantically
different, although the closest neighbor by definition.
We can improve the performance of our AI car by adapting the solution of
the nearest matches using one of several adaptation techniques.
In this light, we build and train a neural network using the top thirty
nearest neighbors of the query problem, and use the solution provided by
the neural network to make our final decision.
\end_layout
\begin_layout Section*
Implementation & Evaluation
\end_layout
\begin_layout Subsection*
Speed-Dreams
\end_layout
\begin_layout Standard
Speed-dreams is an open-source 3D Racing Car Simulator, a fork of the TORCS
project with improved physics and car models/tracks.
We hack this simulator to log data, and also to build our AI bot that plays
the game.
The game source provides a separate module to implement custom AI game
bots, and documents the necessary interface functions that need to be implement
ed for the integration of a custom bot into the game.
\end_layout
\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\align center
\begin_inset Graphics
filename speed-dreams.png
width 50col%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption
\begin_layout Plain Layout
A screenshot of the speed-dreams racing car simulator
\end_layout
\end_inset
\end_layout
\begin_layout Plain Layout
\end_layout
\end_inset
\end_layout
\begin_layout Standard
Our implementation process is multi fold.
In order to build our case base, we first modify the source of an existing
AI bot to log the corresponding game state and control attributes.
We then let the AI bot play over various different racing tracks, and dump
all the required information into a
\emph on
csv
\emph default
.
Once the information has been logged, we use myCBR to import the
\emph on
csv
\emph default
and build the case base.
We experiment with similarity measures using the same software.
For adaptation, we query for the nearest neighbors using the API provided
by myCBR, and implement the neural network using the OpenCV Machine Learning
Module.
\end_layout
\begin_layout Standard
To put it all together, after setting up the above, we build an AI bot that
records the game state parameters at every instant, queries the constructed
case base using the myCBR API to obtain the nearest neighbors, trains a
neural network using these neighbors, and then obtains the final decision
to make for the given game state by feeding it into this neural network.
\end_layout
\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\align center
\begin_inset Graphics
filename /home/abhiram/Documents/academics/8/CS6250 - Memory Based Reasoning in AI/project/fchart1.png
width 50col%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption
\begin_layout Plain Layout
Flow Diagram
\end_layout
\end_inset
\end_layout
\end_inset
\end_layout
\begin_layout Subsection*
Discussion
\end_layout
\begin_layout Standard
Note that we don't learn across cars, but only across tracks.
A major assumption we made is that there are no opponents on the track.
A possible extension of the case base is to include relevant information
about neighboring cars (eg.
distance to nearest car etc.) to make more informed decisions.
We assume that the tracks are of the same category, but different geometric
structure only i.e we learn across different tracks of the same type
\emph on
road
\emph default
as opposed to learning on a road track, and playing on a
\emph on
dirt
\emph default
track.
We spent a significant amount of time in analyzing the influence of various
parameters that the AI car had access to, involving those of the car and
the track.
The attributes in the problem space of the case base are those obtained
after pruning the insignificant parameters.
\end_layout
\begin_layout Standard
\begin_inset Note Note
status open
\begin_layout Plain Layout
To include to to not include control parameters in the problem
\end_layout
\end_inset
\end_layout
\begin_layout Section*
Conclusion
\end_layout
\begin_layout Standard
In this exercise, we experiment with the application of Case-based reasoning
in cruise control for AI Racing cars.
\end_layout
\end_body
\end_document