Architecture of large language models and fuzzy logic control of text generation

<jats:p>The monograph is devoted to the mathematical foundations and software implementation of the architecture of large GPT class language models with fuzzy logic control of text generation. The tokenization of BPE, the mechanism of multi-headed self-awareness, the architecture of the transformer block, pre-training and fine-tuning are considered. A trilogy of fuzzy logic controllers and a FuzzyLLM closed-loop controller have been developed. A 3.1-fold decrease in the decoding entropy has been experimentally confirmed. The accuracy of the spam classifier is 95.67%. The code of 219 cells is reproduced without a GPU. It is intended for graduate students, researchers in the field of machine learning, natural language processing and fuzzy set theory.</jats:p>