Godot is a free and open-source cross-platform game engine for making 2D and 3D games. It provides the developers with a complete toolkit to develop games without restricting their ideas and creativity with a wide variety of in-built features and extensive community support.
Some advantages of Godot Engine over other engines are:
Reinforcement learning (RL) is an interdisciplinary area of machine learning and optimal control concerned with how an intelligent agent ought to take actions in a dynamic environment in order to maximize the cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.
Reinforcement learning differs from supervised learning in not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected. Instead the focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge) with the goal of maximizing the long term reward, whose feedback might be incomplete or delayed.
The environment is typically stated in the form of a Markov decision process (MDP), because many reinforcement learning algorithms for this context use dynamic programming techniques. The main difference between the classical dynamic programming methods and reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the Markov decision process and they target large Markov decision processes where exact methods become infeasible.
Due to its generality, reinforcement learning is studied in many disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, and statistics.
Godot handles the structure of the game using Nodes and Scenes which are placed in a hierarchial tree structure with parent-child relations dictating properties. A Godot game is a tree of scenes and that each scene is a tree of nodes.
Nodes are the fundamental building blocks of the game. There are dozens of kinds that can display an image, play a sound, represent a camera, and much more.
All nodes have the following characteristics:
A name.
Editable properties.
They receive callbacks to update every frame.
Can extend them with new properties and functions.
Can add them to another node as a child.
When nodes are organised in a tree, this construct is called a scene. Once saved, scenes work like new node types in the editor, where they can be added as child for an existing node. In that case, the instance of the scene appears as a single node with its internals hidden.
Scenes allow flexibility in the structure of the game's code. Nodes can be composed to create custom and complex node types, like a game character that runs and jumps, a life bar, a chest with which the player can interact, and more.
The Godot editor essentially is a scene editor. It has plenty of tools for editing 2D and 3D scenes, as well as user interfaces. A Godot project can contain as many of these scenes as you need. The engine only requires one as the application's main scene. This is the scene Godot will first load when a player runs the game.
On top of acting like nodes, scenes have the following characteristics:
They always have one root node
Can save them to your local drive and load them later.
Can create as many instances of a scene as you'd like.
There are different types of Nodes with varying uses and properties. For example, CharacterBody2D node is used for the player character while RigidBody2D node is used for objects like projectiles etc.
All the scenes used to make the game were planned. The scenes include PlayerTank, EnemyTank, Level, Bullet, Bricks, Flags etc
All the required scenes were created independently to be linked later. The scenes were created keeping in mind the abstraction need to link them to othre scenes using Signals and Scripts.
Signals are messages that nodes emit when something specific happens to them, like a button being pressed. Other nodes can connect to that signal and call a function when the event occurs. Signals are a delegation mechanism built into Godot that allows one game object to react to a change in another without them referencing one another. Using signals limits coupling and keeps your code flexible. For example, you might have a life bar on the screen that represents the player's health. When the player takes damage or uses a healing potion, you want the bar to reflect the change. To do so, in Godot, you would use signals.
Tilemaps were used to create the structure of the level scene. Tilemaps are a grid of tiles used to create a game's layout. There are several benefits to using TileMaps nodes to design your levels. First, they make it possible to draw the layout by "painting" the tiles onto a grid, which is much faster than placing individual Sprite2D nodes one by one. Second, they allow for much larger levels because they are optimized for drawing large numbers of tiles. Finally, you can add collision, occlusion, and navigation shapes to tiles, adding greater functionality to the TileMaps.
Scripting in Godot is a crucial aspect of game development within the engine. Godot supports multiple programming languages for scripting, with its own scripting language called GDScript being the primary choice due to its simplicity and seamless integration with the engine's API. For this project, we have chosen GDScript as the scripting language. It is a high-level, dynamically typed programming language specifically designed for game development in Godot. It's similar to Python in syntax and is easy to learn, especially for beginners. GDScript is optimized for Godot's architecture, making it efficient for developing games within the engine.
Signals are a really powerful tool which when used with scripts can get all the logical work done in the Game
The logic related to each node is written in a script connect to the said node. It it used to alter the properties of the node and also to link the node to other nodes and scenes.
Assets such as Sprites for TileMap, tanks, bullets etc were made using Vector Graphics in Inkscape. Inkscape is a free and open-source vector graphics editor for traditional Unix-compatible systems such as GNU/Linux, BSD derivatives and Illumos, as well as Windows and macOS. It offers a rich set of features and is widely used for both artistic and technical illustrations such as cartoons, clip art, logos, typography, diagramming and flowcharting. It uses vector graphics to allow for sharp printouts and renderings at unlimited resolution and is not bound to a fixed number of pixels like raster graphics. Inkscape uses the standardized Scalable Vector Graphics (SVG) file format as its main format.
Rigorous testing using the Remote Scene Tree was done by the team to iron out bugs present in the game to provide a smoother end-user experience and also for non-obstructive implementation of the RL bot.
Reinforcement Learning (RL) is the science of decision making. It is about learning the optimal behavior in an environment to obtain maximum reward. This optimal behavior is learned through interactions with the environment and observations of how it responds. In the absence of a supervisor, the learner must independently discover the sequence of actions that maximize the reward. This discovery process is akin to a trial-and-error search. The quality of actions is measured by not just the immediate reward they return, but also the delayed reward they might fetch. As it can learn the actions that result in eventual success in an unseen environment without the help of a supervisor, reinforcement learning is a very powerful algorithm.
NEAT, or "Neuroevolution of Augmenting Topologies" is a learning algorithm that functions similarly to evolution. In its most basic form, NEAT is a technique for creating networks that are capable of performing a certain activity, like balancing a pole or operating a robot. It’s significant that NEAT networks can learn using a reward function as opposed to back-propagation. By executing actions and observing the outcomes of those actions, an agent learns how to behave in a given environment via reinforcement learning, a feedback-based machine learning technique. The agent receives compliments for each positive activity and is penalised or given negative feedback for each negative action. In contrast to supervised learning, reinforcement learning uses feedback to autonomously train the agent without the use of labelled data. The agent can only learn from its experience because there is no labelled data.
Traditionally, a neural network topology is chosen by a human experimenter, and effective connection weight values are learned through a training procedure. This yields a situation whereby a trial and error process may be necessary in order to determine an appropriate topology. NEAT is an example of a topology and weight evolving artificial neural network (TWEANN) which attempts to simultaneously learn weight values and an appropriate topology for a neural network. In order to encode the network into a phenotype for the GA, NEAT uses a direct encoding scheme which means every connection and neuron is explicitly represented. This is in contrast to indirect encoding schemes which define rules that allow the network to be constructed without explicitly representing every connection and neuron, allowing for more compact representation. The NEAT approach begins with a perceptron-like feed-forward network of only input neurons and output neurons. As evolution progresses through discrete steps, the complexity of the network's topology may grow, either by inserting a new neuron into a connection path, or by creating a new connection between (formerly unconnected) neurons.
For the game, we will be training our algorithm over the player tank. Through repeated generations of evolution and training, the player tank learns the map, along with the positions of all the enemy tanks to shoot them down and gain a maximum possible score.
As with every reinforcement learning algorithm, the agent requires a set of values as an input (the state), which in our case will be the relative positions of the tank with respect to the wall and the nearest enemy tank, the relative position of the flags as well as the direction in which both sets of tanks are pointing.
Godot 4.2.1, Jupyter Notebook, GDScript, Git, Github
We were able to develop a working game fitting our goals of developing a 2D retro game with working AI enemies with endgame handling. The creation of RL bot reached a decent stage which has further scope of improvement.
GameDev and AI Bot Development in Godot 4
Friday, May 10 · 6:00 – 8:00pm
Time zone: Asia/Kolkata
Google Meet joining info
Video call link: https://meet.google.com/yyk-hzsw-hwo
Or dial: (US) +1 205-479-0618 PIN: 279 701 807#
Report prepared on May 6, 2024, 5:14 p.m. by:
Report reviewed and approved by Aditya Pandia [CompSoc] on May 9, 2024, 10:49 p.m..