Titanic Survival Prediction
Description & Thoughts
Given data of the passangers on titanic and their fate, the goal of this project is to predict whether a person will survive the titanic event if we know his/her background. I plan to solve this problem with two approaches: decision tree and neural network.
Decision Tree:
First, I implemented a single decision tree with default hyperparameters and the model achieved 70%-75% accuracy on testing data. Then I used grid search cross validation to tune some hyperparameters and now the model has 75%-80% accuracy on testing data.
Next, I implemented random forest model with default hyperparameters and the model has 75%-80% accuracy on testing data. With grid search hyperparameter tuning, the accuracy increased to around 85%.
During my implementation, I encountered several challenges stated below:
-
Missing data: Currently, I am filling all missing data with 0. It is not the ideal approach, however, because it affects the split of the feature with missing data. Will re-approach this issue later.
-
I am manually dropping features that I believe holds no value(the name of the passenger, passenger ID, ticket number, etc.). However, this will not be feasible when the number of features grow significantly. Plus, my intuition could be wrong. Perhaps PCA would be a solution to eliminate useless features.
-
Depending on how many hyperparameters I am tuning, grid search could take a long time. With better understanding of my data & decision, I could probably decrease the number of hyperparameters for tuning or narrow down the possible values for some parameters. Therefore, the next challenge would be to understand the impact of each hyperparameter on my tree model.
Neural Network:
The initial accuracy rate for my neural network with stochastic gradient descent was around 82%. After tweaking around with some hyperparameters such as initial learning rate, momentum, batch size, and the number of hidden layers, my model achieved 85% accuracy. However, this is still sort of low for a neural network classifier. It is possible that my model is stuck at a local minimum. The next step will be to diagnose if getting stuck at a local minimum is the core of the problem and how can I jump out of it.
Link to GitHub: Click Here
StarCraft 2 Reinforcement Learning Agent
Description
This is a school group project. Our group consists six master's students. The goal of the project is to develop a StarCraft bot using machine learning techniques.
Starcraft II is a real-time strategy game. The win condition for the player in a standard 1 vs. 1 game is to destroy all of opponent's structures. In order to achieve that, the player usually needs to have a stronger army than the opponent. Economy, research, expansion, army composition, army formation, attack timings, scouting, and many other elements can all decide the outcome of the game. The large number of possible actions the player need to decide at each second or even millisecond makes the game quite complex to learn for both humans and computers.
Currently, to reduce complexity, both our agent and the enemy are of the same race and can only perform the same list of pre-defined actions. We also disabled war fog mechanic(which means the agent will be able to fully observe the opponent). Once our agent is more advanced, we will set the opponent to StarCraft's default AI with Hard difficulty. The ideal outcome of the project is to train an agent that can defeat StarCraft's AI with the highest difficulty(Elite). I am responsible for the reinforcement learning approach.
Reinforcement Learning:
q-learning: I defined a small set of actions our agent could perform and selected only a small set of information to be represented as states. With q-learning, our agent achieved 94% win rate against very easy AI. However, the win rate was only 24% percent against easy AI. q-learning works best when state space and action space are relatively small. To defeat higher level AI, I will have to increase the number of actions our agent could perform and also feed more information as states. As a result, q-learning will become too inefficient if not infeasible.
DQN: Instead of having an actual q-table for our agent to look up while making decisions, we could also use neural network to estimate q value while making decisions. To stabilize learning, I implemented two neural networks: one for decision making and one for learning. After each certain amount of game iterations, the two neural network will merge together. I am currently in the training phase of DQN approach.
Challenges & Thoughts:
-
The size of the q-table became a problem when I add more actions for our agent to perform.
-
What reward function will work best for reinforcement learning? StarCraft 2 is a game where gain/loss are not apparent. Except for destroying and losing units, lots of actions that our agent perform will not have immediate impact on the game. Lots of decisions have only long term impact (i.e. research technology or expansion) which makes it hard to judge a decision.
-
Current army control is extremely simple: we directly attack the enemy's base. How could we add more army actions such as formation, chase, kite, surround, defend, etc. Do we need to create a separate model for army control. Decision Tree? Neural Network?
-
How do we decide when to engage the enemy army? Perhaps we can use evaluation functions to judge who has the upper hand.
-
All the build locations are hard coded coordinates. Is there a way to select building location intelligently?
-
How can we deal with cloaked enemy units?
-
I attempted to approach this problem as a computer vision problem. Images are generated based on the current state and the action our agent performed during that state will be the image label. The training result was not ideal due to inconsistent training data. The reason is that StarCraft is a very flexible game. Our agent could make two completely different decision at two different games at the exact same state while both decision led to victory. In other words, our agent could perform different actions on similar images(similar states) and that makes training extremely difficult.
Link to GitHub: Click Here
CIFAR-10 Image Classification
Description & Thoughts
Image classification of CIFAR-10 dataset. The dataset contains over 50,000 images with 10 different classes. Each image has the same size: 32x32.
Convolutional Neural Network:
The CNN I implemented with TensorFlow's Keras API has around 95% accuracy on training data and around 74% accuracy on testing data. I am using maxpooling for the pooling layer and same padding to combat dimensionality reduction. I added one dropout layer to reduce overfitting. I probably need to add more dropout layers as my current model does well on training data compared to testing data.
The first challenge I encountered was how to extract the RGB matrixes from each image and how to link each image with its label(which is stored in a csv file).
The second challenge is to increase accuracy on testing data. The low 74% accuracy could be caused by overfitting or undesirable train/test split(training data contains insufficient number of samples on some classes which makes the model not fit to recognize those classes.)
I will work on the second challenge in my spare time after school assignments.
Link to GitHub: Click Here
Car Evaluation
Description & Thoughts
Customer's value of a car is shaped by the car's attributes such as safety rating, number of doors, maintenance, etc. The goal of this project is to predict a car's value given its attributes. I approached this problem with two methods: first knn and then neural network. Neural network returned desirable results while knn sufferred from accuracy.
Neural Network:
I implemented a neural network classification model with four hidden layers. Two with relu activation function and two dropout layers. Output layer uses softmax. I added two dropout layers to alleviate the problem of overfitting. Even with two dropout layers, the model has around 91% accuracy on training data and 96% accuracy on testing data.
KNN:
When I first started this project, I just finished learning about knn in my machine learning class. Therefore, I wanted to test my knowledge and picked this dataset instead of the iris dataset(the traditional approach for testing knn). However, I only achieved around 24% accuracy on testing data. After I progressed through the machine learning class and learned more, I realized that knn is not a good model for this dataset. The problem knn tries to solve is very similar to clustering problems. The dataset for this project does not share the attributes of a typical clustering problem. Perhaps with PCA, I could narrow down the attributes so that it could be used by knn. But neural network simply outperforms knn. Maybe I would return to this project and use PCA for the sake of practice.
Link to Neural Network: Click Here
Link to KNN: Click Here
Halma Game Playing Agent
Description & Thoughts
This was a school project that aims to develop an AI agent that would play the board game Halma. Minimax is used as the searching algorithm with alpha-beta pruning techiniue to increase efficienty. Below are some of my takeaways from this project:
-
Evaluation function(which requires domain knowledge) is extremely important and it can dictate the performance of the AI agent. For an AI agent to decide where to move, it needs to know how to evaluate the current situation and future consequences of its actions. Therefore, the designed evaluation function directly affects an agent's decision making process.
-
Alpha-beta pruning significantly reduces the time for minimax searching. During mid-game where the number of possible actions has grown quite large, it sometimes took my local machine 30 seconds to 1 min to search a move with 3 levels of depth without pruning. However, with alpha-beta pruning, that number gets reduced to less than 10 seconds or even 5 seconds. Most of the time, it takes less than a second to search a move with pruning.
-
Debug a game playing agent could be challenging. To understand why an agent made a certain move, I had to look at all the search results for that move which was lots of data. It would be even harder to detect a bug or a weakness in an agent because testing an agent in all possible game states is nearly impossible(depending on the game). I discovered several weaknesses of my agent through playing against my friend's agent. But there's no garantee that a hidden bug/weakness would be revealed. Trial and error with lots of testing is the only way that I can think of.
To avoid potential homework plagiarism, no link will be provided. Contact me if you would like to view the code.
Secure Communication
Description
The project establishes a secure communication between server and client using RSA & CCM for data confidentiality & integrity.
Key exchange procedures:
-
The server opens up a socket and the client connects to that socket.
-
The client reads and saves the server’s public key from the file “serverPK.pem”.
-
The client selects a random string as potential secret key and encrypt that key using server’s public key.
-
The client sends encrypted secret key to the server.
-
Upon receiving the encrypted key, the server decrypts the message using its private key.
-
The server encrypts the secret key with client’s public key(accessible to public) and send the encrypted message to client.
-
Client decrypts the message using its private key and compare the plaintext with its version of the secret key. If they are the same, the key establish process is successful. Otherwise, terminates the connection.
-
Simple diagram: Client: Encrypt(secret key, server’s public key) Client Server: Ciphertext Server: Decrypt(Ciphertext, server’s private key) Server: Encrypt(Plaintext, client’s public key) Server Client: Ciphertext Client: Decrypt(Ciphertext, client’s private key) If Plaintext != secret key: client terminates connection.
Communication Procedures: Sender's Perspective:
-
The sender generates some random bytes of length 12.
-
The sender creates a cipher object using the shared secret key and random bytes. The cipher object is in CCM mode using AES encryption.
-
The sender encrypts those random bytes using the cipher object and the encrypted data will be used as the random IV.
-
The sender creates another cipher object using the random IV and the shared secret key. The cipher object is in CCM mode using AES encryption.
-
The sender encrypts the message and generate a tag(used to ensure data integrity) using the cipher object.
-
The sender sends the random IV, encrypted message, and tag to the receiver over the socket.
-
Every time a message is intended to be sent, a random IV(which means a new cipher object) will be generated.
-
Simple diagram: Random bytes = generate(12 bytes) Cipher = new AES(key=shared secret key, nonce=random bytes) Random IV = Cipher.encrypt(Random bytes) Another cipher2 = new AES(key=shared secret key, nonce=random IV) Ciphertext, tag = cipher2.encrypt_and_digest(plaintext) Message = ciphertext + b’@’ + random IV + b’@’ + tag Sender Receiver: Message
Receiver's Perspective:
-
The receiver extract random IV from the received data.
-
The receiver creates a cipher object with the random IV and shared secret key. The cipher object uses CCM mode with AES encryption.
-
The receiver decrypts and verifies the content and integrity of the data with cipher text and tag.
-
If the integrity is compromised, terminates the connection.
-
Simple diagram: Ciphertext, random IV, tag = message.split(b’@’) Cipher = new AES(shared secret key, random IV) Plaintext = cipher.decrypt_and_verify(ciphertext, tag) If MAC check failed, receiver will terminate the connection
Link to GitHub: Click Here