correlr/roam
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
roam/strangeloop-2018.org
Correl d618dea10a Updates
2021-07-29 18:51:25 -04:00

35 KiB
Raw Permalink Blame History

Strange Loop 2018

Itinerary

Tuesday

elm dinner 

  2018/07/25  Small Batch Whiskey and Fare
      Expenses:Food:Lunch  34.91 USD
      Correl

Evening LGBTQ gathering

Wednesday

Richard Feldman

<2018-09-26 Wed 09:00-09:30>

  • Dictionaries as tables

Grant Maki: Facilitating Technology Change Through Cultural Change

<2018-09-26 Wed 09:40-10:10>

  • Trying things in teams
  • Tech lead impostor syndrome
  • Empathizing with fear, skepticism, and choice overload

Very interesting talk from a tech lead position. Grant emphasized building shared experiences, and approaching adoption of new tools and ideas with empathy for the feelings of each team member.

Morning Break

<2018-09-26 Wed 10:10-10:40>

Ravi Chugh: Structured Editing for Elm in Elm

<2018-09-26 Wed 10:40-11:10>

  • "Sketch-n-Sketch"

  • Structured code editor "Deuce"

    • Shows AST on code
    • Apply context-aware transformations to the AST
  • Type inspector mode
  • Supports holes
  • Live feedback for incomplete programs

  • https://hazel.org - Holes for incomplete code (Cyrus Omar)

    • Friday Strangeloop talk

Anthony Deschamps: Programming Robots with Elm

<2018-09-26 Wed 11:30-12:00>

EV3DEV
Debian distribution for LEGO Mindstorms
?
Robotics is about perception, behavior, and control
PLatform.Worker
Application sans view function

Lunch

<2018-09-26 Wed 12:00-13:00>

Food trucks!

  2018/09/26  The Cheese Shac
      Expenses:Food:Lunch  10.20 USD
      Correl

Matthew Griffith: Building a Toolkit for Design

<2018-09-26 Wed 13:00-13:30>

  • Legos

    • Everything fits together
    • Easy to build
    • Easy to change

  • CSS

    • Dependent on lots of external factors
    • Lots of gotchas
  • mdgriffith/elm-ui (Website)

  • Palettes

    • Consists of

      • Colors
      • Spacing
      • Typography
    • A concise set of style rules (~50 max)
  • Each property defines its own tests, which can be run in arbitrary environments
  • eightshapes :: Design systems

I'm eager to discuss the material in this talk with UI folks when I get back. There are likely many parallels with the design team's styleguide CSS framework and the ideas presented here and in the related materials.

Dillon Kearns: Extending Type Safety Beyond Elm's Borders

<2018-09-26 Wed 13:40-14:10>

Afternoon Break

<2018-09-26 Wed 14:10-14:40>

Sam Row: Complex Animations Done Well

<2018-09-26 Wed 14:40-15:10>

  • Easings

    • Tracks starting and target state
    • Animation updates will be jarring if the target state changes

  • Springs

    • Tracks current position and velocity, and target state

  • Decoupling animation logic from business logic

    • Separate models for application state and visual state

Alley Kelly McKnight: Naming Things in Elm

<2018-09-26 Wed 15:20-15:50>

  • Categories for naming rules

    • Naming conventions
    • Guiding lights

  • What makes a name good?

    • Does the name keep us in the process flow?
    • Does the name help the reader build an accurate mental model of the code?
    • Does the reader know accurately what the thing you're naming is?
  • The Mental Model: Our Human Narrative

The idea of using names to allow functions to flow as a narrative is an interesting one, as is maintaining a consistent metaphor throughout.

Tereza Sokol: Demystifying Parsers

<2018-09-26 Wed 16:00-16:30>

  • elm/parser

  • Includes support for building useful error messages

    • problem : String -> Parser a

I've enjoyed using Haskell's Parsec library in the past, as well as boost::spirit in C++ back when I first dipped my toes into EBNF grammars. This will be interesting to play with when I can come up with a reason to 😊.

Food Trucks

<2018-09-26 Wed 17:00-19:00>

  2018/09/26  Union Station Restaurant & Lounge
      Expenses:Food:Dinner  23.60 USD
      Correl

Strange Loop Party

<2018-09-26 Wed 19:00-21:30>

Thursday

Thursday Breakfast

<2018-09-27 Thu 08:30-09:00>

Shaping our children's education in computing

<2018-09-27 Thu 09:10-10:00>

  • Project quantum (questions)
  • Teals (high schools)
  • Csteachers org
  • Raspberry Pi foundation
  • Education is rife with unintended consequences - be humble, cooperative

I'm curious how the resources mentioned in this talk could be applied to AWeber's school outreach programs.

Zero Downtime Migrations of Stateful Systems

<2018-09-27 Thu 10:20-11:00>

  • Three different types of migrations

    • perceived zero Downtime
    • actually zero Downtime
    • no migration of state

  • Perceived Zero Downtime

    • Migrating existing records

      • constant syncing
      • incremental

    • Building confidence

      • automated reconciliation and checks
      • row counts
      • test environment

    • Decouple interactions

      • allow reads
      • replay writes

    • Migration

      • switch writes to queue
      • wait for sync to complete
      • switch database
      • replay queue

  • Actual zero Downtime

    • load snapshot
    • dual writes
    • metrics on any issues

  • No migration

    • new service built aware of legacy system
    • older data expires
    • run in tandem
    • delegate to legacy system when needed
    • test with dual writes

    • monitoring

      • unexpected behaviors
      • absent expected state
    • slow roll-out

These are excellent bullet-points to consider with all of our migrations. It'll be good to have this sort of plan prepared for any project that requires one.

Chasing the Stream Processing Utopia

<2018-09-27 Thu 11:20-12:00>

  • Architecture

    • Ingestion (various sources)
    • Processing (using Apache Samza)

  • Complexity

    • Stateless processing (filtering, data enrichment)

    • Stateful processing (aggregation)

      • Windowing
      • Beam processing model

  • Accuracy

    • Exactly once processing

      • Ingestion (de-dupe)
      • Pipelined processing (Chandy Lamport)
      • Storing results (idempotent writes / de-dupe)

  • Scale

    • Scaling ingestion

      • Managed by cloud services, or yourself when self-hosting

    • Scaling processing

      • Challenges

        • horizontal scaling
        • limited to least performant component

      • Typical bottlenecks

        • data access

          • accessing adjunct data (databases, services)
          • managing temporary state
        • CPU
    • Any source (including batch sources, HDFS)
    • Any language

    • Anywhere

      • Stream processing as a service

    • Awesome tools

      • Schema management

        • Schema evolution
      • ACL management
      • Topic management
      • Stream Processing Job Management
      • Data Lineage (e.g. Apache Atlas)

      • Monitoring

        • Ingestion (e.g. Kafka Monitor)
        • Job Progress (e.g. Burrow)

While a good deal of the content was technology-specific, mapping out the data flow of some of our services and seeing how they may fit in a stream processing environment would be interesting.

Thursday Lunch

<2018-09-27 Thu 12:00-13:30>

  2018/09/27  Lyft to Pappy's
      Expenses:Travel:Taxis  6.46 USD
      Expenses:Travel:Taxis:Tip  2.00 USD
      Correl

  2018/09/27  Lyft from Pappy's
      Expenses:Travel:Taxis  6.48 USD
      Expenses:Travel:Taxis:Tip  2.00 USD
      Correl

Really good BBQ. Enjoyed some ribs, brisket, and burnt ends.

Rosie Pattern Language

<2018-09-27 Thu 13:30-14:10>

  • Reasoning

    • Regular expressions do not scale
  • Combing through data to get devops insights
  • libraries of named and namespaced expressions

  • RPL syntax looks like a programming language

    • reads/diffs like code
    • executable unit tests
    • macros
  • RPL expressions compose

  • RPL is based on parsing expression crammers, which are greedy and possessive

    • makes it difficult to be accidentally inefficient
    • Allows parsing recursive grammars
  • Logstash (Grok) uses pattern libraries

  • Can your 'grep' do this?

    • curl -s www.google.com | rosie grep -o subs net.url_common

      • Prints all matching URL sub-expressions

    • sed -n 46,49p /var/log/system.log | rosie match all.things

      • all.things represents a disjunction of a variety of things, different pattern matches will be color-coded

    • head -n 1 /var/log/system.log | rosie grep -o jsonpp num.denoted_hex

      • JSON-structured output

  • Formal basis

    • Chomsky hierarchy

      • regular - context-free - context-sensitive - recursively enumerable
      • RPL (and all PEG grammars) are context-sensitive

  • Shares some similarities with regular expressions

    • repetition (same syntax, greedy and possessive)
    • character sets (simplified syntax, one name or list or range at a time)
    • Operations (look-ahead, negation, etc.) (simplified syntax)
    • "choice" differs, / is ordered choice, possessive)
  • Ships with a standard library

  • Debugging

    • Trace a mismatch

      • echo '17:30:4' | rosie trace time.rfc3339
    • REPL!

  • Uses

    • IBM uses rosie in a big-data pipeline
    • Mining source code repositories
    • NCSU students wrote RPL patterns to extract language features from different languages (including bash!)
    • White-listing valid input (protecting against injection attacks per OWASP)

    • Python bindings! (also, c, go), more coming

      • Not very "pythonic", currently

  • Resources

I really like the idea of pattern libraries and reuse (doing it with regular expressions is really hard), and the ability to specify full grammars is very nice. I'm curious whether this could be integrated into our validation to avoid unnecessary duplication.

Architecting for Active-Active Resiliency in the Cloud

<2018-09-27 Thu 14:30-15:10>

  • Data and Service resiliency and durability

  • Basics (Ingredients)

    • Availability (normal operation time / total time)

      • mean time between failure (MTBF) / mean time to repair (MTTR)

      • Multiple components in series (x and y) (A = Ax * Ay)

        • Availability decreases

      • Multiple components in parallel (multiple services x) (A = 1 - (1 - Ax)^2

        • Availability increases

      • Basic Active Architecture

        • Web -> App -> DB

          • cheap, simple, fast to market
          • manual intervention, low resiliency, high potential rework

      • Basic Active-Passive Architecture

        • Load balanced app with fail-over nodes / read-replica db
        • Hot or cold standby
        • decreased downtime, - backup services & data, good balance
        • increased costs, complexity, potential for data loss

      • Basic Active-Active Architecture #1

        • Multiple instances taking traffic simultaneously
        • always on, minimum data loss, zero intervention
        • limited tools, most complex, most expensive

  • Considerations & trade-offs (Appetizers)

    • Data Replication

      • Active-Active (avoid this)

        • multiple master databases synchronizing

          • collisions
          • latency

      • Active-Passive

        • Multiple instances writing to a single data store

          • promote a read replica on failure
          • Active-Active at the service level, Active-Passive at the data level

      • Easy data replication databases (managed)

        • Cloud Spanner
        • Cosmos
        • Aurora

      • Easy data replication databases (not managed)

        • Apache Cassandra
        • MySQL

    • Considering tenants - what are we failing over?

      • Similar to considering restart strategies in OTP supervision trees
    • As availability and durability increases, so does complexity and cost

  • Resiliency patterns (The Entrees)

    • Circuit breaker (stop retrying)
    • Exponential back-off (slow down retries)
    • Fallback (error response or cache)
    • Multiple instances in different availability zones within a region
    • Worker queues and streams decouple components
    • Patterns can be combined

  • Tools and services (The Desserts)

    • Chaos Monkey
    • Serverless

It's important to prepare for failure, and balancing availability and consistency is always a balancing act.

For failure, I like to map out services and their dependency as I would an Erlang supervision tree, drawing boundaries around tightly coupled components, and defining the failure strategy for each group.

Towards Language Support for Distributed Systems

<2018-09-27 Thu 15:30-16:10>

Research is very animated! (doesn't look like it from the outside)

  • Language support for distributed system = ways that the programming language/compiler can aid in the development of a distributed application

  • Some areas of research

    • Consistency & Programming Models

      • Consistency?

        • Specifically programming models that provide some kind of consistency guarantee (e.g, what guarantees do I have for a replica of an object/datum?)
        • Sometimes weak consistency is good enough

      • Conflict-free replicated data types (CRDTs)

        • Lasp: A Language for Distributed, Coordination-Free Programming
      • Spry is a programming model for building applications that want to trade-off availability and consistency at varying points in application code to support application requirements (CA in CAP)

      • Mixed consistency via "Correctables" (Incremental Consistency Guarantees for Replicated Objects)

        • Give the user different APIs depending on the sort of consistency they're after

          • invokeWeak(operation)
          • invokeStrong(operation)
          • invoke(operation[, levels])

      • Mixed consistency via "MixT" (MixT: A Language for Mixing Consistency in Geodistributed Transactions)

        • Answers the question of what if we want to choice from multiple consistency options in one application

    • Session Types

      • Can be though of as types for communication protocols

      • Many variants of session types

        • Binary & Multiparty (# of parties)
        • Static & Dynamic
      • Introduction to session types by Simon Fowler

      • Prototypes of session types are implemented for many languages

        Erlang
        monitored-session-erlang
        Python
        SPY
        Haskell
        effect-sessions
        Scala
        Ichannels
        Java
        Scribble
    • Static Analysis & Verification

    • Others

    • Birds-eye view of fringe projects

  • Resources

Keynote: Erica Joy Baker - Changing the World

<2018-09-27 Thu 16:30-17:20>

  • Passionate about talking about diversity and inclusion
  • It seems that everyone wants to "change the world"
  • "We did it! … Mission accomplished, in the words of George Bush. We changed the world"
  • "We got so focused on disruption that we didn't stop to think about who or what we were disrupting"
  • Recommends watching "Halt and Catch Fire"

  • "Silicon Valley is Venture Capital"

    • Story of "The Scorpion and the Frog"
    • The VCs are the scorpions "in case that wasn't clear".

  • Steps to change the world

    1. Have an idea
    2. Grow your ideas slowly.
    3. Get customers
    4. Get some more of them
    5. Keep going
    6. Build what you want to build on your own terms.
  • Mailchimp is a sponsor (bleh), has no VC funding
  • We have to ask ourselves, "I can do this, but should I do this?"
  • "How we're going to change the world is by the legacy we leave behind."
  • "We must build companies that represent everyone, and all experiences."
  • Call to action: If you work in a company that you know is doing something harmful, that you have averted your eyes from, now is the time to speak up and do something.
  • Change the world, just make sure you're changing it for the better.

Dinner 

  2018/09/27  Schlafly Tap Room
      Expenses:Food:Dinner  19.06 USD
      Correl

Evening 

  2018/09/27  Union Station Restaurant & Lounge
      Expenses:Food:Drinks  11.17 USD
      Correl

  2018/09/27  Union Station Restaurant & Lounge
      Expenses:Food:Drinks  12.78 USD
      Correl

Friday

Friday Breakfast

<2018-09-28 Fri 08:30-09:00>

All the Languages Together

<2018-09-28 Fri 09:10-10:00>

  • Different languages have different strengths

  • Multi-Language software is hard

    • FFI/JNI
    • microservices

  • Can we allow reasoning about interop at the source level?

    • As opposed to running code and encountering errors / debugging

  • Language specifications are incomplete - don't account for linking

    • 'escape hatches' (ML/C FFI, Rust/unsafe, Java/JNI)
    • We tell programmers to "be careful"

  • Rethink PL Design: Principled FFIs

    • Design linking types extensions that support safe interoperability with other languages

      • linear types (similar to rust ownership)
      • pure (terminating) functions
    • Only need linking type extensions to interact with behavior/features inexpressible in your language
    • Linking Types for Multi-Language Software: Have Your Cake and Eat It Too

    • Reasoning about refactoring

      • Let the programmer specify, do I want to only link with pure code? Allow linking with stateful code?

    • Minimal annotation burden

      • Provide a default translation (e.g, a pure language defaults to linking pure code, not stateful)

  • Catching cross-language type errors

    • What happens if the linked code doesn't have linking type annotations?

    • Type-preserving compilation

    • Cross-language type errors

      • Fail linking if target (compiled) types don't match

  • Refactoring-Preserving Compilation

    • Also known as equivalence-preserving compilation, fully abstract compilation, and secure compilation
    • Specifying a non-default linking type means giving up some refactoring patterns
    • Programmer gets to decide what refactorings matter to them; LT annotations tell the compiler what linking to allow/disallow
    • Linking types are the escape hatch

  • Ongoing work

    • Mulberry Project

      • Building linking types to interop ML with Rust
      • Building linking types for a pure language
      • Building on top of web assembly (generating a richly typed Wasm with state/effect encapsulation)

    • What about untyped languages (Scheme, C)?

      • Untyped Wasm (gradual typing)
      • control linking type
  • SILC - Secure Interoperability, Languages & Compilers

Leverage vs. Autonomy in a Large Software System

<2018-09-28 Fri 10:20-11:00>

  • Big company problems

    • How do you make good use of hundreds of software engineers to build a large system?
    • How do you organize a large software system so hundreds of engineers working on it can avoid wasting their time?

  • Andy grove on org structure (High Output Management)

    • functional team - increased leverage

    • mission-oriented team - increased speed

      • speed is the only advantage

  • Conway's Law

    • Software tends to look like the organization that built it

  • Coplein's Law

    • If your software doesn't look like your organization, you've got trouble

  • Leverage vs Autonomy

    • centralized systems - increased leverage
    • decentralized systems - increased autonomy
    • poor leverage - waste time reinventing wheels
    • poor autonomy - waste time waiting on other teams

  • Costs of autonomy/decentralization

    • difficult to make global improvements
    • difficult to make cross-cutting changes
    • uncontrolled system complexity (little consistency)

  • Costs of leverage/centralization

    • single points of failure / monoculture risk
    • arcade knowledge (downside of knowledge specialization)
    • uncontrolled system complexity (poor modularity)

  • sweet spot: cluster scheduling

    • increased leverage, removed repetitive operational work
    • most customers want roughly the same thing
    • no important loss of autonomy
    • global system improvement (upgrades)
    • bugs (very rarely) bring down multiple services

  • sweet spot: finagle as theft service framework

    • high leverage
    • most customers want roughly the same thing
    • no important loss of autonomy
    • difficult to debug

  • sour patch: monolithic service

    • lots of autonomy problems (major development choke-point)
    • poor sense of ownership
    • very easy to cause cross-cutting bugs

    • single ci/deploy pipeline

      • easy to make cross-cutting changes
    • good leverage

  • sweet spot: service oriented architecture

    • autonomy is really good
    • every project needs its own ci/deployment pipeline
    • difficult to cause cross-cutting bugs
    • good ownership
    • code is very modularized
    • leverage is low
    • difficult to make cross-cutting changes

  • sour patch: cross-service application logic library

    • where to put cross-cutting logic? attempted a shared library, embedded in a variety of separate services
    • low autonomy (lots of services to deploy with changes)
    • bad ownership (which team owns this?)

  • sour patch: unowned services

    • consequence of autonomy

      • services get lost in the shuffle between teams

  • sour patch: ad-hoc service platform

    • building stuff into existing services, rather than creating a new one

      • caused by an aversion to build a whole new service
      • "What's the fastest way I can get this piece of work done?"
    • owners of the code get a lot of leverage, but lose a lot of autonomy

  • Strato: platform for microservices

    • monolithic service that hosts services inside of it
    • captures some common patterns (e.g., a data store with a cache in front of it, etc.)
    • Goal is to split the systems concerns from the applications themselves (e.g., timeouts, retries, etc.)
    • Microservices within are isolated, allows modularity
    • a lot easier for the code structure to match the organization structure

  • Thrift RPC

    • communication between services is structurally typed
    • "compatibility" relation between types makes upgrades easier

  • StratoQL: a DSL for microservices

    • structurally typed
    • native support for thrift data
    • transparent concurrency (cf. Haxl (previous Sl talk))

    • large gain in leverage over separate microservices

      • captures common infrastructure patterns in configuration
    • autonomy for application logic owners
    • many customers want roughly the same thing
    • complicated interface
    • centralized team a bottleneck
    • difficult to debug
    • downtime affects many microservices