Artificial Intelligence: Type of environments

Let’s first describe what is an agent in artificial intelligence. An intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals. Intelligent agents may also learn or use knowledge to achieve their goals. They may be very simple or very complex.

When designing artificial intelligence solutions we need to consider aspects such as the the characteristics of the data (classified, unclassified, …), the nature of learning algorithms (supervised, unsupervised, …) and the nature of the environment on which the AI solution operates. We tend to spend big amounts of time in the first two aspects but it turns out, that the characteristics of the environment are one of the absolutely key elements to determine the right models for an AI solution. Understanding the characteristics of the environment is one of the first tasks that we need to do. From this point of view we can consider several categories.

Fully vs Partial observable

An environment is called fully observable if what your agent can sense at any point in time is completely sufficient to make an optimal decision. For example, we can imagina a card game where all the cards are on the table, the momentary site of all those cards is really sufficient to make an optimal choice.

An environment is called partialy observable where you need memory on the side of the agent to make the best possible decision. For example, in the poker game the cards are not openly on the table, and memorizing past moves will help you make a better decision.

Deterministic vs Stochastic

A deterministic environment is one where your agent’s actions uniquely detemine the outcome. For example, in the chess game there is really no randomness when you move a piece, the effect of moving a piece is completely predetermined and, no matter where I am going to move the same piece, the outcome is the same.

A stochastic enviroments there is a certain amount of radomness involved. Games that involve a dice, are stochastic. While you can still deterministically move your pieces, the outcome of an action also involves throwing the dice, and you cannot predict it.

Discrete vs Continuous

A discrete environment is one where you have finitely many action choices, and finitely many things you can sense. For example, the chess has finitely many board positions and finately many things you can do.

A continuous environment is one where the space of possible actions or things you could sense may be infinite. In the game of dards, throwing a dard we have infinite ways to angle it and accelerate it.

Benign vs Adversarial

In benign environments, the environment might be random, it might be stochastic, but it has no objective on its own that would contradict the own objective. Weather is benign, it might be ramdon, it might affect the outcome of your actions but it is not really out there to get you.

In adversarial environemnts, the opponent is really out to get you. In the game of chess the enviroment has the goal of defeat you. Obviously, it is much harder to find good actions in adversarial environments where the opponent actively observes you and counteracts what you are trying to achieve than in benign environments.

I have seen a few more classifications or specifications but, more or less, all of them list the same categories or very similar categories.

Note: Article based on my notes of the course Intro to Artificial Intelligence | Udacity

Artificial Intelligence: Type of environments

Machine learning vs deep learning

AI (artificial intelligence) has been one of the buzzwords in the recent years and, it looks like it is going to continue been like that for, at least, a few more. Leaving on a side terminologies, the truth is that this kind of technologies offer great opportunities. Lately, there are two terms that we can listen quite often and almost everywhere. These two terms, related with AI, are:

  • Machine learning
  • Deep learning

Both terms are related, in fact, we can say the machine learning involves deep learning. Both technologies refer systems that can learn by themselves, the difference is the way they learn. As a quick explanation, we can say that deep leaning is more complex, more sophisticated and more autonomous. Once the deep leaning system is implemented the need for human intervention is minimal.

Machine learning

The main characteristic that differentiate these systems from other less advanced is the ability to learn by themselves. In this way, the system algorithm receives a set of rules to apply to the data but, the special thing about this kind of system is they can adapt these rules or develop new ones to increase the successful rate.

For example, let’s say we write a system to identify cat pictures (Internet loves cat pictures, we know). We can ask the system to identify some patterns: four legs, hair, nose, ears, tail, two eyes… All characteristics that usually cats have. After that, we can train the algorithm with a training set pointing to the system if we can find a cat or not. With this action we allow the system to create or adapt their own rules to make easier the task when new and unknown pictures will be given.

Deep learning

The difference between machine learning and deep learning is that the second one takes the learning part to a more advanced level. In this case, the system has layers or neuronal units trying to imitate the brain’s behavior.

In deep learning, each layer process the information and return a result as a percentage. For example, this picture has a 87% change to be a cat and a 13% change to not. The next layer analyzing the image will take this value and it will combine it with its own value. With this, the percentage will vary and, this new value will be sent to the next layer to perform a similar process. This process will continue layer after layer.

All these consecutive analysis performed by the different layers reduce the error rate and increase the number of correct conclusions. To train the system we will use again a training set and, specially in this case, the bigger, the better.

Machine learning vs deep learning