Decoding AI Jargons With Chai

I have started learning GANAI from Chaicode.com. In our first lecture, we discussed the foundation of AI.

We learned about Google's 2017 article titled “Attention Is All You Need.”

We studied the Transformer model architecture. These are some of the topics I am learning:

1. Transformer - Model Architecture

2. Encoder

   In the encoder, when we give input to any AI model, it is known as a prompt. This input then goes through a process called tokenization, where the input language or text is converted into numbers. Each word (or subword) in the input is assigned a number based on the AI model's algorithm — such as Byte Pair Encoding (BPE) used in models like ChatGPT.

   Example:

    Input: "I love AI"
    After Tokenization (BPE): [34, 87, 56] (Each number represents a token from the vocabulary.)The cat sat on the mat
    Generate Token is 
    [976,9056,10139,402,290,2450] // example
   

3. Decoder

   When tokens are converted back into text, the AI model uses a decoder. The decoder takes all the tokens and generates the final output (human-readable text).

    Input:
    The cat sat on the mat
   
    Generated Tokens (Example):
    [976, 9056, 10139, 402, 290, 2450]
   
    Decoded Output:
    The cat sat on the mat
   

4. Vector Embeddings:

Semantic Meaning

Positional Encoding helps the AI model understand the position of each word in a sentence — because the meaning of a sentence depends not just on the words, but how they are arranged.

1. Positional Encoding

   Positional encoding like there name what position given all input that vise dicided what sentence means

   Example 1: Same words, different order

   • The cat sat on the mat

   • The mat sat on the cat

These two sentences use the same words, but the order is different, so the meaning changes completely.
Without positional encoding, the model might treat them as the same.
But with positional encoding, the model knows:

So, positional encoding ensures the model understands who is doing what.

1. Example 2: Same word, different meaning

   • The river bank is peaceful.

   • The RBI bank is secure.

Even though it’s the same word, the meaning changes depending on:

• The context (surrounding words like “river” or “RBI”)

• The position of those words in the sentence

This is how the model figures out the correct meaning using both semantic value and position.

2. Self-Attention

   All tokens talk with each other because it helps decide their meaning.

3. Multi-Head Attention

4. Temporal Knowledge Cutoff

   It means the AI model does not have information about what is currently happening in the world — this is called knowledge cutoff.

5. Vocabulary Size
   The total number of tokens a model can recognize is called its vocabulary size.

6. Temperature

   When we use the AI model in the playground or implement it in our application, there's an option called temperature, with a value between 0 and 2.
   When the temperature is high, the answers become more random and creative.
   When the temperature is low, the answers are shorter and more focused or accurate.