Open Source 2023 Notes

Inside ChatGPT

• Lab651 and Recursive Awesome, Captovation - AI Solutions Development
• Applied AI - 501c3 organization

GPT Releases

• Version 1 - 117M parameters
• Version 2 - 1.5B parameters
• Version 3 - 175B parameters
• Version 4 - 1T+ parameters

Wolfram One

• Cloud/desktop hybrid, integrated computation platform
• Similar to Jupyter notebooks
• Lets you play around with GPT models (older than v3 and v4)

GPT 2

• Always using the next highest probability word isn't always the best... using a "temperature" parameter, the model can be tweaked to sometimes choose words that aren't quite the highest probablity, leading to fewer run-on sentences.

• Attention - when the model picks up on a higher probability and begins to focus in on it

• Embeddings - training model on relationships between words (word pairs in a sense)

  • Example: Munich - Berlin + France => Paris
  • This can also be done for sequences of words and whole blocks of text

• ChatGPT has learned syntactic grammar using embedding techniques

  • When determing a sequence of words, it essentially follows a random walk through the syntactic grammar

• Wolfram can break down the structure

• OpenAI playground: https://platform.openai.com/playground

• "How would you create a neural network model that would rule the world? Please write it out in small, specific steps."

Stable Diffusion

• "Speaking plainly" to computers -isn't this programming?
• Hannah Loegering - @fractalorange
• Open Source AI:
  • Deforum (Video, Image)
  • Automatic1111/stale-diffusion-webai
  • Langchain
  • Simulacra and Simulation
  • GPT4All Chat (text)
  • Flowise/langchainJA-ui
• @secondsightvisuals
• Music is a 4-dimensional dataspace
• Replicate - lucid sonic dreams, takes a source and produces an infinite sequence
• phygital+ - create 3d-printed models
• Can download stale diffusion and run locally
• Midjourney runs over discord?
• Luma AI - create 3D scenes and scan from 2D video
• Kaiber - super easy AI animation
• Wonder Dynamics - 3D character repalcement in 2D video
• Deep Motion - 3D character rigging from 2D video
• Runway ML - Full suite of video tools including their Gen1 and Gen2
• Blockade Labs: text to 360 image
• Simulacra - build 3D assets

Role of Generative AI in Healthcare

• CTO of Virtuwell (Telehealth)
• Virtuwell is not-for-profit?!

Care Delivery

• Diagnosis prediction
• Support clinicians by providing relevant medical info
• Personalized health and wellness advice to individuals

Medicine

• Personalized medicine: Forecast outcomes via patterns
• Drug Discovery: identify possible new meds and predict compounds likely to be effective

Health Plan

• Organize unstructured medical data

Virtuwell AI initiatives

• Reduce over/under staffing by predicting the future
• Predict surge traffic to website
• Analyze patient data for systematic follow ups
• Funneling relevant medical records
• At-home kit for colon cancer screening and STI

Healthcare Companies in AI

• Optellum: rules out harmless cases of lng canver via scans.
• Ultromics: increases diagnostic accuracy of coronary heart disease from 80% to 94%
• IDx-DR (now Digital Diagnostics): correctly identifies diabetic retinopathy by 87.4% (8% better than industry standard)
• Beyond Verbal: Detecting Coronary Artery Disease by listening to voice

Future of Generative AI

• If you are a large enterprise, you need to train your own LLM

Under-Engineering

• Information / Technology impedance
  • Programmers making wrong asusmptions based on limited informaiton
  • Tribal knowledge lost over time and developer turnover
  • Lack of documentation
    • Code not self-documenting
    • Unreadable unit tests
    • Distance between code and documentation (documentation is on a remote server away from source)
• Self-inflicted complexity
  • Ineffective bootstrapping: programmers bringing in their own toolsets instead of assessing what tools are best for their projets
  • Programmers building their own frameworks
    • Overfitting
  • Bug-fixes / rework
    • Bandaging code quickly instead of reworking code complexity
• Under-engineering: act of explicitly engineering in a simplistic manner
• Architecture should be as simple as possible
• Developer documentation should be simple and accessible (developers should have their own set of documentation)
• Coding should be simple, readable, and maintainable

Reduce Complexity

• Find the most confusing concepts o fthe codebase
• Document Confusion
• Unit tests should be your guide on refactors
  • Unit tests should be user friendly
• Reverse code entropy - deleting code??
• Identify:
  • Challenging developer cadences
  • confusing folder structures
  • Hard-to-read / hard-to-write code
  • Bad model or variable names
• Put documentation next to code
• Design for NOW, but with the future in mind
• Ask what-if, but don't design for what-if in order to ensure you don't box yourself out
• Push back on product feature work
• Use 90% rule for generic/inheritance patterns
• Provide code samples
• Write comments first? (why not write unit tests first? :P)

IOT Eventing

General Workflow

Devices -> Event Ingestion -> Event Matching -> Event Delivery -> Event Actions -> Back to Device(s)

Design Constraint: Event Volume

• Devices/Locations = 100
• Max Events per Device: 10 events/s
• SLO: < 1s response time from client controllers to devices

Evolution of Eventing

• 2011 - Monolithic, event-based architecture built around RabbitMQ
• 2013 - Monolith + sharding
• 2015 - Microservices: strong REST principles instead of Event Oriented Architecture, using Kafka instead of RabbitMQ
• 2015-2019 - Event pipeline: Kafka, Amazon EC2, SQS, oh my...
• 2019-current: one suscription system instead of a subscription per device type
  • Move to a stateful subscription model
  • One users events will always go through the same box
  • Makes latency stable
• Breaking down and logically dividing up work wihtin a system is critical