Microsoft

Python

MALLET

Chat GPT

DEWEY can leverage Machine Learning and Large Language Models

1.  Repository : https://huggingface.co/
1.  The Bloke : https://huggingface.co/TheBloke
1.  Lone Striker : https://huggingface.co/LoneStriker
1.  WebGUI : https://github.com/oobabooga/text-generation-webui
1.  Stable Diffusion : https://github.com/AUTOMATIC1111/stable-diffusion-webui
1.  Voice Changer : github.com/w-okada/voice-changer
1.  Real Time Voice : https://github.com/RVC-Project/Retrieval-based-VOice-Conversion-WebUI
1.  RVC : voice-models.com  and weighs.gg

Amazon Sage Maker

1.  Canvas
    1. Low Code.  Drag and Drop.
1.  Studio
1. Code and Models

Used for creating LLM in AWS.
AI to determine bank loans.

Amazone BedRock RAG Application

1. Boto3
2. LangChain
3. PyPDFLoader
4. FAISS
5. Streamlit

Other AI Engines to Explore

1. Stable Diffusion (Stability) : https://stablediffusionweb.com/ or civitai.com
2. Watson (IBM) : https://www.ibm.com/products/watson-explorer
3. Chess
4. Content Hub. IBM Watson can propose relevant tags based on content.
5. Bard/Palm 2 (Google)
6. Google blog post about BERT,18 an ML technique for NLP, the benefit shown was simply the ability to link a preposition with a noun.
7. Aladin (BlackRock)
8. Mindjourney (MindJourney) : https://www.midjourney.com/home/?callbackUrl=%2Fapp%2F
9. Kaaros
10. Tensor Flow (Google)
11. IRIS : https://iris.ai/
12. Claude https://www.anthropic.com/index/claude-2
13. https://marketplace.atlassian.com/apps/1224655/scrum-maister?hosting=cloud&tab=overview
14. Bing (free)
15. Claude 2 (free) by Anthropic
16. Grok by X (Twitter)
17. Open-source models (FREE) available on Huggingface https://huggingface.co/
18. Llama 2 by Meta
19. Flan, Falcon, Orca, Beluga, Mistral, Mixtral, Phi2
20. LMStudio (Windows, Mac, Linux) - install and run models
21. Pinokio.computer browser - install and run models
22. Atlassian Rovo - https://www.atlassian.com/blog/announcements/introducing-atlassian-rovo-ai

References

1. PubMED : https://pubmed.ncbi.nlm.nih.gov/30153250/
2. Other :https://www.aiforlibrarians.com/ai-cases/
3. Science Direct : Weed Collections : https://www.sciencedirect.com/science/article/pii/S0099133317304160?via%3Dihub
4. Apache Mahout : https://mahout.apache.org//
5. Spark MLlib Apache : https://spark.apache.org/mllib/
6. Stanford : https://library.stanford.edu/blogs/stanford-libraries-blog/2022/07/working-students-library-collections-data
7. M-Files. Smart subjects provide tag suggestions based on document content
8. Magellan's AI capabilities include speech and text analytics from contextual hypothesis and meaning deduction.
9. AWS Innovate: Data and AI/ML Edition
10. Data Science : https://www.dataplusscience.com/GenerativeAI.html
11. AI Got Talent : https://dataplusscience.com/files/UCCBAGenAI20240206.pdf

Regulation

MEPs substantially amended the list to include bans on intrusive and discriminatory uses of AI systems such as:

1. “Real-time” remote biometric identification systems in publicly accessible spaces;
2. “Post” remote biometric identification systems, with the only exception of law enforcement for the prosecution of serious crimes and only after judicial authorization;
3. Biometric categorisation systems using sensitive characteristics (e.g. gender, race, ethnicity, citizenship status, religion, political orientation);
4. Predictive policing systems (based on profiling, location or past criminal behaviour);
5. Emotion recognition systems in law enforcement, border management, workplace, and educational institutions; and
6. Indiscriminate scraping of biometric data from social media or CCTV footage to create facial recognition databases (violating human rights and right to privacy).

AI Feedback Loop

• Programming : Algorithm + Input => Answers
• Supervised Learning : Answer + Input = > Algorithm
• Feedback Loop : Re-perturbed feeds classification and classification feed perturbed
• Decision Tree : mathematically produced else
• Neural Network : binary tree diagrams
• Natural Language Processing (NLP_ : the identification of patterns in spoken or written text.
  • Read Understand Derive meaning from Human Languages
  • Lanaguage Structures
  • INteract Transfer Data
  • Feed Document -> encode -> segmentation into sentences by punctuations. words in the sentences into constiutainet words into tokens. tokenize. remove no$
  • ALgorithm
  • Explain (skip, skipping skipped _ same stemming.
  • limitization lemmatizaition
  • verbs particle - speech tagging
  • Pop culture references movies places news locations- named entitity tagging

Segmentation into sentences and store them. Tokenizing into words and store them. Remove Stop Words (Non Essential Words.) like are, and, the from sentence segments. Stemming treats skip, skipping, skipped as the same words. Lemmetization Am Are Is for all genders ich bin, du bist, er/sie/es ist base word (lemma)= be Named Entity Tagging of Proper Nouns Sentiment and Speech with naive bayes

• Reinforcment Learning : data - > model
• Cumulative Selection : Building off the last step. Not starting over everyt ime
• Semi-Supervisied Machine Learning : Supervised means there is some human involvement in setting up the tool,
• Inductive Reasoning
  • The corpus-based approach using a training set, as described above, uses the process of inductive reasoning. This is the kind of thinking that states ‘the sun rose yesterday, the sun rose today, so the chances are the sun will rise tomorrow’. Now, philosophers will argue that inductive reasoning is not scientific. Just because the sun rose yesterday does not mean the sun will rise tomorrow.

AI Terms

• A Corpus : All text documents in Scholar

• A Training Set : is a subset of the corpus, which has been tagged in some way to identify the characteristic you are looking for Common Crawl RefinedWeb The Pile C4 Starcoder BookCorpus o ROOTS Wikipedia o Red Pajama

• A Test Set : collection of documents to be used for trialling the algorithm, to see how successfully it carries out the operation.

  • Example : Modified National Institute of Standards and Technology (MNIST) database of handwritten numbers,10

• An Algorithm : The ‘algorithm’ is simply the tool that looks at each item in the corpus and enables a decision to be made. An algorithm may be as simple (and frequently is as simple) as matching a pattern. selecting and applying an algorithm or method

AI Process Cycle

1. Identify
2. Explore / Analyze / Encode (Change Everything into a number)
3. Model
4. Predict
   1. Clarity (Makes Sense)
   2. Original (Novelty)
   3. Useful
5. Feedback

AI Methodology

1.Evolutionary Algorithms (EAs) Evolutionary Algorithms are a subset of optimization algorithms inspired by the process of natural selection and genetics. They are used to solve complex optimization problems by evolving solutions over generations. Key concepts and components include:

Population: A set of candidate solutions. Selection: The process of choosing the fittest individuals based on a fitness function. Crossover: Combining parts of two or more solutions to create offspring. Mutation: Introducing random variations to solutions to maintain genetic diversity. Fitness Function: A measure of how well a solution solves the problem. EAs are particularly useful for problems where the search space is large, complex, or poorly understood. They have applications in areas such as engineering design, scheduling, and machine learning model optimization.

1.Deep Reinforcement Learning (DRL) Deep Reinforcement Learning combines reinforcement learning with deep learning. It involves training agents to make decisions by interacting with an environment to maximize cumulative rewards. Key components include:

Agent: The entity making decisions. Environment: The world with which the agent interacts. State: A representation of the environment at any given time. Action: Choices available to the agent. Reward: Feedback from the environment based on the action taken. Policy: A strategy used by the agent to decide actions based on states. Value Function: Estimates the expected return of states or actions. DRL is particularly powerful for tasks involving sequential decision-making, where the environment is dynamic and complex. Applications include robotics, game playing (e.g., AlphaGo), autonomous vehicles, and financial trading.

Relationship and Differences Evolutionary Algorithms focus on optimization through population-based search and are often used for static optimization problems. Deep Reinforcement Learning emphasizes learning optimal policies through interaction with dynamic environments, leveraging neural networks to handle high-dimensional state spaces.

Resources https://www.youtube.com/watch?v=awGJkRe9m50