November 29th, 2023

A core component of learning is separating signal from noise on a given subject. The corollary is that a core component of teaching a subject is informing the student which parts are noise; the signal falls out from this.

November 1st, 2023

Having dealt with C# DLL hell for years, and then going through the equally hellish dotnet HttpClient fiascos, and now struggling with adopting ES modules in a Typescript library, I am beginning to believe that Microsoft has a department focused on making module resolution a living hell for anyone who uses their tools.

Solid Freeform 2023 Notes

From August 14-16, 2023 I attended an academic conference on 3D printing research, below are my notes.

Technion Additive Manufacturing Center

• Current geometric design pardigm ha sbeen with us for over a decade
• V-rep operations:
  • Filleting and rouding
  • Mixed symbolic/numeric computations over multivariate (and V-rep) splines
• https://csaws.cs.technion.ac.il/~gershon/irit

In-Situ Process Monitoring in AM

Michael J. Heiden

Purpose, Motivation

• Failure mode analysis (ie spike in oxygen)
• Reduce coupon inspection
• Runs in parallel with the build process

Detection Methods and Processes

• Optical IR
• Use the Peregrine machine learning model from Oakridge
• Use multi-modal data (thermal imaging/thermography, high speed cameras, spectroscopy, acoustic monitoring)

Using the Data to Create Useful 3D Digital Twins

• Show discrepancies between what is supposed to be printed and the actual

Future Targets

• Real time monitoring
• Closed lop control for defect correction (this is what we do!)

Machine Learning Applied to Process Monitoring for Laser Hot Wire Directed Energy Deposition

Carnegie Mellon University

Comparison with Laser Powder Bed Fusion

• Under acceptable conditions, melt pools are not stable
• Monitoring of rare unwanted events such as spatter requires significant ML traning (1k-10k hand-labeled images)

Laser Hot Wire Additive Manufacturing

• Well behaved melt pools under normal conditions
• Can monitor entire buildds in real time
• Approach: detect unsteady behavior of the melt pool, don't attempt to learn the appearnce of irregular events

Methodology

• Many flaws exist across multiple frames, so a time dependent analysis is beneficial
• Long Short-Term Memory networks are able to learn time dependencies in video data
• Use previous frames to predict future frames (about 10-20 frames of past history)
• Capture a regularity scorePredicition: within 5-10 years we will see fully sensed and automated DED systems

Known Flaw Types

• Stable Deposition
• Melt pool oscillation
• Arcing
• Wire stubbing
• Wire dripping

Summary and Conclusions

• ML model detects unsteady behavior, which is indicative of process instability (unlike LBPF)
• Prediction is that DED systems will be deployed with fully automatic quality control in next 5-10 years

Area Printing

• Area printing is stamping on the powderbed
• Seurat is the company that developed this technology

Rethinking Additive Manufacturing

• DfAM transcends disciplinary boundaries
• Traditional manufacturing informs our design sensibilities
• Evaluate design using consensual design techniques
• DfAM increases as experience increases
• Restrictive DfAM: "Can we print it"
• Opportunistic DfAM: "Should we print it"
• Unlearning requires designers to identify good DfAM solutions
• VR can help with DfAM
• Use VR for simulating a printer in test?
  • Probably not super beneficial for our needs

Co-deisgn of 3D Printing

• You cannot monitor your way out of trouble: need all the insights possible from in-situ and operando monitoring.
• In aircraft manufacturing: "Anything that allows you to gain efficiency is worth the manufacturing cost"
• Diffusion Modeling is not sufficient for AM Heat Exchangers
  • Use heat exchangers used with solar energy?
  • Alternate technology: diffusion bonding
• Improved heat exchangers reduces defects?

Physical Validation of Job Placement Optimization in Cooperative 3D Printing

• Traditional vs Cooperative 3D printing
  • Traditional: gantry systems
  • Limited in scalability and flexibility
• Partitioning Strategy: Chunking
  • First chunk into layers (Z-Chunking)
  • XY-Chunking: chunk adjacent chunks at an angle to facilitate bonding of chunks
• Use grid search algorithm (A* Path Planning) to plan robot movement
  • If there are two robots that are going to take a path at the same time, apply a conflict avoidance algorithm
• AMBOTS C3DP
• When a robot moves to a new location, it requires manual calibration

Time Optimal Path Planning for Heterogenous Robots in Swarm Manufacturing

• Continuous spatial profile with arbitrary robot gemoetry that includes orienatation
• Dynamic environment with arbitrary number of obstacles
• Time-optimal solution for multiple agents
• Projecting Dynamic Obstacles
  • Use "safe intervals" to reduce number of computations
  • Based on safe intervals, split the configurations based on what has more than one safe interval
• Limitations:
  • Does not make real-time adjustments
  • Truly centralized optimal planning scales exponentially with the number of agents
• Open questions:
  • How does shape discretizaiton impact solution quality?
  • What is the upper number of agents before it is too computationally expensive?

Layer Wise Prediction of Microstructural Evolution

• Predict grain size and melt-pool depth using thermal model features
• As structure grows, cooling rate decreases
• Prediction using physics-guided machine learning (SVM models)
  • Predict Meltpool Depth
    • Not inuitively explained by process parameters
  • Thermal Feature Correlate to Grain Size
    • Grain size predicted with 90% F-Score
• Cooling rate also decreases with heat build-up

Post Superficial Temperature Monitoring During Additive Manufacturing

• CNN goal is to produce predicted temperature of build
• Best results come from combining a machine learning model, in situ monitoring, and sensors
• Normal Data + Wall Simulation as inputs gives best ML performance on experimental data
• In Situ Sensors + ML models as a "Digital Twin" of how the part will be produced

Optimal Control of Wire DED Systems

• Modeling of a dynamic system usually using ODE
• Optimal Control Theory - https://en.wikipedia.org/wiki/Optimal_control

Framework for Physics Guided Machine Learning

• Process variations are high in AM
• PSP: Process-structure-property

PSP Linkages for In-situ Data

• Predictive Melt-pool Modeling
  • Multi-layer model produced best results
• Predictive Pore-structure Model
• Predictive Surface-height Model

AI Driven In-Situ Monitoring

Intrinsic Keyhole Oscillation

Keyhole: a puncture in the powderbed that leads to an unexpected cavity ("keyhole pore") in the powderbed

• Combine X-Ray imaging and Thermal Imaging
• Intrinsic Keyhole Oscillation
  • Surface tension and coil pressure
  • Liquid flow on the outside of the meltpool increases keyhole size
  • Lower than 30KHz
• Perturbative Keyhole Oscillation
  • Greater than 40KHz

Gradient-weighted Class Activation Mapping (Grad CAM)

• Predict where keyholes will form using an ML model, avoiding further X-Ray synchotron experiments
• An input to the trained model is a simulation of the build

PRISM: Process Parameter Optimization for Selective Manufacturing

• Goldilocks zone:
  • Not enough: lack of fusion
  • Too much: keyhole formulation
• Use machine learning to determine proper process parameters!
• Goal is to achieve high density, assume density is a surrogate for:
  • CTE (Coefficient of thermal expansion)
  • Elastic modulus
  • Strength
  • Toughness
• Data will be published and open source
• ML Driven Design of Experiment
  • Latin hyper cube sample generation
  • Predict Printability using tree algorithm (bounds determined by expert)
  • Predict density using tree algorithms (XGBoost)
• Select candidates: diversity (determined by L2 distance), quality, and random sleection
• Used FLAML to automate experimentation
• Would probably need to re-train per material type

FAIR Knowledge Management System for Additive Manufacturing

• AMMD.nist.gov, AMBench2022.nist.gov
• ASTM F42 - working group developing data models to represent AM

Unconventional Data Sources for Market Intelligence

• Aachen Center for AM
• Market intelligence uses external data over internal data (patent databases, social media, etc.)
• Databases used for research: Amadeus, LinkedIn, Scopus

Digital Twins

• Part qualification is manual and slow
• Can digital twins help understand the process to manufacturing property map?
• Direct Ink Write
• Using robotic automation to generate large amounts of parts and data

Executable for Avant Garde Laser Exposure (EAGLE)

• Common command interfaces
• Ability to support any input file
• Plugin support
• Process Buckets
• Limited interference with machine
• Common Build Data Storage
  • SQL Database on machine or network
• Print Process Flow:
  • Pre-Process
  • Layer Process:
    • Verify -> Scan -> Characterize -> Recoat
  • Post-Process
• EAGLE is written in Rust and Javascript using Tauri
  • Designed to support print files > 100GB

Direction not intention determines destination

In-Situ Monitoring Using Neutron Rays

• Neutrons have high penetration in metals
• Different contrast from X-Rays
• Phase evolution
• In-Situ Laser Metal Deposition
  • Completmentary to lab and synchotron experiments
• In-Beamline Strain Measurement
• Neutrons are non-destructive, tri-axial, and sub-surface
• Additive has a much wider distribution of gamma prime and gamma double prime sizes

X-Ray Compute Tomagrphy

• Determine powder size because that's always going to be stuck on the surface
• Could minimum powder size (and laser amplitude?) define minimum defects size?

SATURN

SCT and MPM Data

• Lack of fusion and keyhole are two major issues (laser power)
• Rnadom forest prediction works with decent acccuracy
• U-net model?
• Some defects will heal automatically, and don't need to be treated
• Non-balanced gain control introduces saturation in the photodiodes
• OEM gives a graphical representation of the data, not a tabular or formatted version
• Photodiodes operate at >11MHz
• Sensor captures images at 400um resolution

Intelligent Process Control

• Using FPGA to provide faster response times
• Goal is to catch a defect within the transient state (10-20us), they've (potentially) achieved nanosecond response times

High precision measurement of Melt Pools

• FPS of 40,000
• Resolution: 896 x 176 pixels

End-to-End AI Models for Error Detection

• Current issue - solutions are not general
• Matta (Cambridge company), Grey-1 AI Copilot for AM
• Multi-head residual attention convolutional neural network:
  • Develops its own masks to determine where to pay attention

Automated Project Publishing

The idea: a small script that periodically runs, checking for changes to project readme's, and publishes updates to davidvedvick.info.

This script would need to reside in a-personal-site. Would need to be configurable so that it could accurately be run on a local machine but also on a build server.

Update 11/28/2023: I instead implemented this as GoCD pipeline that I run on-demand, and it is glorious.

July 30th, 2023

Programming Proficiency

To solve programming problems reliably one needs two skills:

• Proficiency in set theory.
• Specifying the correct behavior of code in tests.

Organization (naming things, type hierarchies, module placement) is also extremely useful for ensuring code maintainability, but it is a tangential skill to the first two skills.

June 9th, 2023

The pareto rule strikes again - employees are at their best when they're working at 85% capacity, not 100% - https://hbr.org/2023/06/to-build-a-top-performing-team-ask-for-85-effort.

May 28th, 2023

Most dependencies are just data. Data "repositories" are easier to fake than mock. Maybe mocks should just be used for hardware I/O?

Open Source 2023 Notes

Inside ChatGPT

• Lab651 and Recursive Awesome, Captovation - AI Solutions Development
• Applied AI - 501c3 organization

GPT Releases

• Version 1 - 117M parameters
• Version 2 - 1.5B parameters
• Version 3 - 175B parameters
• Version 4 - 1T+ parameters

Wolfram One

• Cloud/desktop hybrid, integrated computation platform
• Similar to Jupyter notebooks
• Lets you play around with GPT models (older than v3 and v4)

GPT 2

• Always using the next highest probability word isn't always the best... using a "temperature" parameter, the model can be tweaked to sometimes choose words that aren't quite the highest probablity, leading to fewer run-on sentences.

• Attention - when the model picks up on a higher probability and begins to focus in on it

• Embeddings - training model on relationships between words (word pairs in a sense)

  • Example: Munich - Berlin + France => Paris
  • This can also be done for sequences of words and whole blocks of text

• ChatGPT has learned syntactic grammar using embedding techniques

  • When determing a sequence of words, it essentially follows a random walk through the syntactic grammar

• Wolfram can break down the structure

• OpenAI playground: https://platform.openai.com/playground

• "How would you create a neural network model that would rule the world? Please write it out in small, specific steps."

Stable Diffusion

• "Speaking plainly" to computers -isn't this programming?
• Hannah Loegering - @fractalorange
• Open Source AI:
  • Deforum (Video, Image)
  • Automatic1111/stale-diffusion-webai
  • Langchain
  • Simulacra and Simulation
  • GPT4All Chat (text)
  • Flowise/langchainJA-ui
• @secondsightvisuals
• Music is a 4-dimensional dataspace
• Replicate - lucid sonic dreams, takes a source and produces an infinite sequence
• phygital+ - create 3d-printed models
• Can download stale diffusion and run locally
• Midjourney runs over discord?
• Luma AI - create 3D scenes and scan from 2D video
• Kaiber - super easy AI animation
• Wonder Dynamics - 3D character repalcement in 2D video
• Deep Motion - 3D character rigging from 2D video
• Runway ML - Full suite of video tools including their Gen1 and Gen2
• Blockade Labs: text to 360 image
• Simulacra - build 3D assets

Role of Generative AI in Healthcare

• CTO of Virtuwell (Telehealth)
• Virtuwell is not-for-profit?!

Care Delivery

• Diagnosis prediction
• Support clinicians by providing relevant medical info
• Personalized health and wellness advice to individuals

Medicine

• Personalized medicine: Forecast outcomes via patterns
• Drug Discovery: identify possible new meds and predict compounds likely to be effective

Health Plan

• Organize unstructured medical data

Virtuwell AI initiatives

• Reduce over/under staffing by predicting the future
• Predict surge traffic to website
• Analyze patient data for systematic follow ups
• Funneling relevant medical records
• At-home kit for colon cancer screening and STI

Healthcare Companies in AI

• Optellum: rules out harmless cases of lng canver via scans.
• Ultromics: increases diagnostic accuracy of coronary heart disease from 80% to 94%
• IDx-DR (now Digital Diagnostics): correctly identifies diabetic retinopathy by 87.4% (8% better than industry standard)
• Beyond Verbal: Detecting Coronary Artery Disease by listening to voice

Future of Generative AI

• If you are a large enterprise, you need to train your own LLM

Under-Engineering

• Information / Technology impedance
  • Programmers making wrong asusmptions based on limited informaiton
  • Tribal knowledge lost over time and developer turnover
  • Lack of documentation
    • Code not self-documenting
    • Unreadable unit tests
    • Distance between code and documentation (documentation is on a remote server away from source)
• Self-inflicted complexity
  • Ineffective bootstrapping: programmers bringing in their own toolsets instead of assessing what tools are best for their projets
  • Programmers building their own frameworks
    • Overfitting
  • Bug-fixes / rework
    • Bandaging code quickly instead of reworking code complexity
• Under-engineering: act of explicitly engineering in a simplistic manner
• Architecture should be as simple as possible
• Developer documentation should be simple and accessible (developers should have their own set of documentation)
• Coding should be simple, readable, and maintainable

Reduce Complexity

• Find the most confusing concepts o fthe codebase
• Document Confusion
• Unit tests should be your guide on refactors
  • Unit tests should be user friendly
• Reverse code entropy - deleting code??
• Identify:
  • Challenging developer cadences
  • confusing folder structures
  • Hard-to-read / hard-to-write code
  • Bad model or variable names
• Put documentation next to code
• Design for NOW, but with the future in mind
• Ask what-if, but don't design for what-if in order to ensure you don't box yourself out
• Push back on product feature work
• Use 90% rule for generic/inheritance patterns
• Provide code samples
• Write comments first? (why not write unit tests first? :P)

IOT Eventing

General Workflow

Devices -> Event Ingestion -> Event Matching -> Event Delivery -> Event Actions -> Back to Device(s)

Design Constraint: Event Volume

• Devices/Locations = 100
• Max Events per Device: 10 events/s
• SLO: < 1s response time from client controllers to devices

Evolution of Eventing

• 2011 - Monolithic, event-based architecture built around RabbitMQ
• 2013 - Monolith + sharding
• 2015 - Microservices: strong REST principles instead of Event Oriented Architecture, using Kafka instead of RabbitMQ
• 2015-2019 - Event pipeline: Kafka, Amazon EC2, SQS, oh my...
• 2019-current: one suscription system instead of a subscription per device type
  • Move to a stateful subscription model
  • One users events will always go through the same box
  • Makes latency stable
• Breaking down and logically dividing up work wihtin a system is critical

Drag And Drop Difficulties

It is surprisingly difficult to get drag and drop working in jetpack compose. I have had it working in most parts of the list for most of the week, but the first element causes the LazyList to readjust its bounds, causing absolute chaos. A fix I discovered in Google's examples partially helps, but still leaves the dragged element in limbo:

April 13th, 2023

Good source code requires other developers know as few assumptions as possible.

March 12th, 2023

Fixing an Annoying Connection Issue

Currently really trying to nail down initializing a connection to a media server with project blue, so that it is portable and hopefully transitions easily to Jetpack Compose and navigation. I wonder what the best way is to initialize connection state? Here's my requirements:

• If initializing the connection fails, need to send user to settings.
• If no media server connection is selected, send user to the settings.
• If initializing succeeds, continue normal execution.

This piece of code has also caused me numerous, hard to track issues for years, so if I could make as much of it as testable as possible, then this would be a win. My current idea is to do this:

• Have an interface ControlConnectionInitialization that defines a single function, promiseInitializedConnection(libraryId: LibraryId): ProgressingPromise<BuildingConnectionStatus, IConnectionProvider?>, through delegation, I can build up different behaviors as needed.
• Have a simple implementation of this interface that just proxies the connection:

class ConnectionInitializationProxy(
    private val manageConnectionSessions: ManageConnectionSessions,
) : ControlConnectionInitialization {
    override fun promiseInitializedConnection(libraryId: LibraryId): ProgressingPromise<BuildingConnectionStatus, IConnectionProvider?> =
        ProgressingPromiseProxy(manageConnectionSessions.promiseLibraryConnection(libraryId))
}

• Have another implementation of ControlConnectionInitialization that handles error state and routes the user to the settings in failure. This will be passed an inner ControlConnectionInitialization, which we will assume is handling the basic connection (and maybe success case).
• At the top level, have another implementation of ControlConnectionInitialization which handles the normal execution states, and delegates error-handling to the error-handling implementation.

I finished this work in this pull request.