1027 lines
35 KiB
Org Mode
1027 lines
35 KiB
Org Mode
:PROPERTIES:
|
|
:ID: 521281f0-c051-4ef0-87e1-7776ea9f836e
|
|
:END:
|
|
#+TITLE: Strange Loop 2018
|
|
#+STARTUP: indent
|
|
#+PROPERTY: header-args:ledger :exports none
|
|
#+OPTIONS: toc:nil num:nil tags:nil prop:("ADDRESS")
|
|
#+COLUMNS: %ITEM %TIMESTAMP
|
|
#+LATEX_COMPILER: pdflatex
|
|
#+LATEX_HEADER: \usepackage[final]{pdfpages}
|
|
#+LATEX_CLASS: tufte-book-chapters
|
|
#+LINK: github https://www.github.com/
|
|
#+LINK: gitlab https://www.gitlab.com/
|
|
#+LINK: elm https://package.elm-lang.org/packages/%s/latest
|
|
#+filetags: :talks:
|
|
|
|
#+BEGIN_EXPORT latex
|
|
%% Notes
|
|
\newenvironment{NOTES}
|
|
{\begin{marginfigure}}
|
|
{\end{marginfigure}}
|
|
|
|
%% Code Blocks
|
|
\definecolor{codebg}{rgb}{0.95,0.95,0.95}
|
|
\setminted{bgcolor=codebg}
|
|
|
|
%% Quotes
|
|
\newcommand*\openquote{\makebox(25,-22){\scalebox{5}{``}}}
|
|
\newcommand*\closequote{\makebox(25,-22){\scalebox{5}{''}}}
|
|
|
|
\definecolor{shadecolor}{rgb}{0.95,0.95,0.95}
|
|
|
|
\BeforeBeginEnvironment{quote}{\begin{snugshade}}
|
|
\AtBeginEnvironment{quote}{\openquote}
|
|
\AtEndEnvironment{quote}{\hfill\closequote}
|
|
\AfterEndEnvironment{quote}{\end{snugshade}}
|
|
#+END_EXPORT
|
|
|
|
* Travel :noexport:
|
|
** Flight
|
|
- Record Locator :: WHIBQL
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/08/06 American Airlines
|
|
Expenses:Travel:Flights 334.88 USD
|
|
Expenses:Travel:Flights:Taxes 53.52 USD
|
|
AWeber
|
|
|
|
2018/09/25 American Airlines -- Checked bag
|
|
Expenses:Travel:Flights:Luggage 25.00 USD
|
|
Correl
|
|
|
|
2018/09/25 Taxi to hotel
|
|
Expenses:Travel:Taxis 42.14 USD
|
|
Correl
|
|
|
|
2018/09/28 Taxi from hotel
|
|
Expenses:Travel:Taxis 46.64 USD
|
|
Correl
|
|
|
|
2018/09/28 American Airlines -- Checked bag
|
|
Expenses:Travel:Flights:Luggage 25.00 USD
|
|
Correl
|
|
|
|
2018/09/28 Philadelphia Parking Authority
|
|
Expenses:Travel:Parking 44.00 USD
|
|
Correl
|
|
#+END_SRC
|
|
*** Departing
|
|
:PROPERTIES:
|
|
:ADDRESS: PHL
|
|
:END:
|
|
<2018-09-25 Tue 14:31>--<2018-09-25 Tue 15:53>
|
|
- Carrier :: American Airlines operated by PSA Airlines as American Eagle
|
|
- Flight Number :: 5644
|
|
- Seat :: 20C (Economy)
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/25 Philly Pretzel Factory
|
|
Expenses:Food:Lunch 8.49 USD
|
|
Correl
|
|
#+END_SRC
|
|
*** Returning
|
|
:PROPERTIES:
|
|
:ADDRESS: STL
|
|
:END:
|
|
<2018-09-28 Fri 13:25>--<2018-09-28 Fri 16:39>
|
|
- Carrier :: American Airlines
|
|
- Flight Number :: 1777
|
|
- Seat :: 13D (Economy)
|
|
** Hotel
|
|
:PROPERTIES:
|
|
:ADDRESS: 1820 Market St., St Louis, MO 63103
|
|
:END:
|
|
<2018-09-25 Tue 16:00>--<2018-09-28 Fri 11:00>
|
|
- Hotel :: St. Louis Union Station Hotel
|
|
- Confirmation Number :: 3469659829
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/07/16 Union Station Hotel Confirmation
|
|
Expenses:Travel:Hotel 600.00 USD ; 3 nights @ 200.00/night
|
|
Expenses:Travel:Hotel:Taxes 113.58 USD
|
|
AWeber
|
|
#+END_SRC
|
|
|
|
* Ticket :noexport:ATTACH:
|
|
:PROPERTIES:
|
|
:Attachments: strange-loop-2018-ticket-6DTV-1.pdf
|
|
:ID: 44AAE8B1-99BF-4F20-A1DE-8BB031861A13
|
|
:END:
|
|
|
|
- Customer :: Heather Geiger
|
|
- Company Name :: AWeber
|
|
- Registration Reference :: 6DTV
|
|
- Invoice Number :: A20180687
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/06/14 Strange Loop 2018
|
|
Expenses:Conference:Strange Loop 725.00 USD
|
|
Expenses:Conference:elm-conf 125.00 USD
|
|
AWeber
|
|
#+END_SRC
|
|
|
|
#+BEGIN_EXPORT latex
|
|
\includepdf[pages=1-1]{data/44/AAE8B1-99BF-4F20-A1DE-8BB031861A13/strange-loop-2018-ticket-6DTV-1.pdf}
|
|
#+END_EXPORT
|
|
* Itinerary
|
|
** Tuesday
|
|
*** elm dinner
|
|
:PROPERTIES:
|
|
:ADDRESS: Small Batch - Whiskey and Fare
|
|
:END:
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/07/25 Small Batch Whiskey and Fare
|
|
Expenses:Food:Lunch 34.91 USD
|
|
Correl
|
|
#+END_SRC
|
|
|
|
**** Company :noexport:
|
|
- Jim Carlson
|
|
- Jeremy Fairbank
|
|
- Jason Dew
|
|
*** Evening LGBTQ gathering
|
|
:PROPERTIES:
|
|
:ADDRESS: Union Station Hotel
|
|
:END:
|
|
|
|
**** Company :noexport:
|
|
- Greg Walker
|
|
- Noah Gordon
|
|
- Richard Feldman (briefly)
|
|
** Wednesday
|
|
|
|
*** Richard Feldman :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 09:00-09:30>
|
|
|
|
- Dictionaries as tables
|
|
*** Grant Maki: Facilitating Technology Change Through Cultural Change :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 09:40-10:10>
|
|
|
|
- Trying things in teams
|
|
- Tech lead impostor syndrome
|
|
- Empathizing with fear, skepticism, and choice overload
|
|
|
|
#+BEGIN_NOTES
|
|
Very interesting talk from a tech lead position. Grant emphasized
|
|
building /shared/ experiences, and approaching adoption of new tools
|
|
and ideas with empathy for the feelings of each team member.
|
|
#+END_NOTES
|
|
*** Morning Break
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 10:10-10:40>
|
|
*** Ravi Chugh: Structured Editing for Elm in Elm :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 10:40-11:10>
|
|
|
|
- "Sketch-n-Sketch"
|
|
- Structured code editor "Deuce"
|
|
- Shows AST on code
|
|
- Apply context-aware transformations to the AST
|
|
- Type inspector mode
|
|
- Supports holes
|
|
- Live feedback for incomplete programs
|
|
- https://hazel.org - Holes for incomplete code (Cyrus Omar)
|
|
- Friday Strangeloop talk
|
|
*** Anthony Deschamps: Programming Robots with Elm :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 11:30-12:00>
|
|
|
|
- [[https://www.ev3dev.org][EV3DEV]] :: Debian distribution for LEGO Mindstorms
|
|
- Robotics is about perception, behavior, and control
|
|
- =PLatform.Worker= :: Application sans view function
|
|
*** Lunch
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 12:00-13:00>
|
|
|
|
Food trucks!
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/26 The Cheese Shac
|
|
Expenses:Food:Lunch 10.20 USD
|
|
Correl
|
|
#+END_SRC
|
|
|
|
**** Company :noexport:
|
|
- Alex Koppel
|
|
- Rain
|
|
- Noah Gordon
|
|
- Michael Donatz
|
|
*** Matthew Griffith: Building a Toolkit for Design :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 13:00-13:30>
|
|
|
|
- Legos
|
|
- Everything fits together
|
|
- Easy to build
|
|
- Easy to change
|
|
- CSS
|
|
- Dependent on lots of external factors
|
|
- Lots of gotchas
|
|
- [[elm:mdgriffith/elm-ui][mdgriffith/elm-ui]] ([[https://elm-ui.netlify.com/][Website]])
|
|
- Palettes
|
|
- Consists of
|
|
- Colors
|
|
- Spacing
|
|
- Typography
|
|
- A concise set of style rules (~50 max)
|
|
- Each property defines its own tests, which can be run in arbitrary
|
|
environments
|
|
- eightshapes :: Design systems
|
|
|
|
#+BEGIN_NOTES
|
|
I'm eager to discuss the material in this talk with UI folks when I
|
|
get back. There are likely many parallels with the design team's
|
|
styleguide CSS framework and the ideas presented here and in the
|
|
related materials.
|
|
#+END_NOTES
|
|
|
|
*** Dillon Kearns: Extending Type Safety Beyond Elm's Borders :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 13:40-14:10>
|
|
- Addressing "avoidable uncertainty"
|
|
- GraphQL :: [[github:dillonkearns/elm-graphql][dillonkearns/elm-graphql]]
|
|
- Generates code from a specification
|
|
- TypeScript :: [[github:dillonkearns/elm-typescript-interop][dillonkearns/elm-typescript-interop]]
|
|
- Generates TypeScript type declarations for elm ports and flags
|
|
- Slides & Resources :: https://bit.ly/typeswithoutborders
|
|
*** Afternoon Break
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 14:10-14:40>
|
|
*** Sam Row: Complex Animations Done Well :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 14:40-15:10>
|
|
|
|
- Easings
|
|
- Tracks starting and target state
|
|
- Animation updates will be jarring if the target state changes
|
|
- Springs
|
|
- Tracks current position and velocity, and target state
|
|
- Decoupling animation logic from business logic
|
|
- Separate models for application state and visual state
|
|
*** Alley Kelly McKnight: Naming Things in Elm :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 15:20-15:50>
|
|
|
|
- Categories for naming rules
|
|
- Naming conventions
|
|
- Guiding lights
|
|
- What makes a name good?
|
|
- Does the name keep us in the process flow?
|
|
- Does the name help the reader build an accurate mental model of
|
|
the code?
|
|
- Does the reader know accurately what the thing you're naming is?
|
|
- The Mental Model: Our Human Narrative
|
|
|
|
#+BEGIN_NOTES
|
|
The idea of using names to allow functions to flow as a narrative is
|
|
an interesting one, as is maintaining a consistent metaphor
|
|
throughout.
|
|
#+END_NOTES
|
|
*** Tereza Sokol: Demystifying Parsers :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Centene
|
|
:END:
|
|
|
|
<2018-09-26 Wed 16:00-16:30>
|
|
|
|
- [[elm:elm/parser][elm/parser]]
|
|
- Includes support for building useful error messages
|
|
- =problem : String -> Parser a=
|
|
|
|
#+BEGIN_NOTES
|
|
I've enjoyed using Haskell's Parsec library in the past, as well as
|
|
=boost::spirit= in C++ back when I first dipped my toes into EBNF
|
|
grammars. This will be interesting to play with when I can come up
|
|
with a reason to 😊.
|
|
#+END_NOTES
|
|
*** Food Trucks
|
|
:PROPERTIES:
|
|
:ADDRESS: Union Station - 20th St
|
|
:END:
|
|
<2018-09-26 Wed 17:00-19:00>
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/26 Union Station Restaurant & Lounge
|
|
Expenses:Food:Dinner 23.60 USD
|
|
Correl
|
|
#+END_SRC
|
|
*** Strange Loop Party
|
|
:PROPERTIES:
|
|
:ADDRESS: City Museum
|
|
:END:
|
|
<2018-09-26 Wed 19:00-21:30>
|
|
** Thursday
|
|
*** Thursday Breakfast
|
|
<2018-09-27 Thu 08:30-09:00>
|
|
*** Shaping our children's education in computing :session:keynote:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre
|
|
:END:
|
|
<2018-09-27 Thu 09:10-10:00>
|
|
|
|
- Project quantum (questions)
|
|
- Teals (high schools)
|
|
- Csteachers org
|
|
- Raspberry Pi foundation
|
|
- Education is rife with unintended consequences - be humble,
|
|
cooperative
|
|
|
|
#+BEGIN_NOTES
|
|
I'm curious how the resources mentioned in this talk could be applied
|
|
to AWeber's school outreach programs.
|
|
#+END_NOTES
|
|
*** Zero Downtime Migrations of Stateful Systems :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Pepsi Encore
|
|
:END:
|
|
<2018-09-27 Thu 10:20-11:00>
|
|
|
|
- Three different types of migrations
|
|
- perceived zero Downtime
|
|
- actually zero Downtime
|
|
- no migration of state
|
|
|
|
- Perceived Zero Downtime
|
|
- Migrating existing records
|
|
- constant syncing
|
|
- incremental
|
|
- Building confidence
|
|
- automated reconciliation and checks
|
|
- row counts
|
|
- test environment
|
|
- Decouple interactions
|
|
- allow reads
|
|
- replay writes
|
|
- Migration
|
|
- switch writes to queue
|
|
- wait for sync to complete
|
|
- switch database
|
|
- replay queue
|
|
- Actual zero Downtime
|
|
- load snapshot
|
|
- dual writes
|
|
- metrics on any issues
|
|
- No migration
|
|
- new service built aware of legacy system
|
|
- older data expires
|
|
- run in tandem
|
|
- delegate to legacy system when needed
|
|
- test with dual writes
|
|
- monitoring
|
|
- unexpected behaviors
|
|
- absent expected state
|
|
- slow roll-out
|
|
|
|
#+BEGIN_NOTES
|
|
These are excellent bullet-points to consider with all of our
|
|
migrations. It'll be good to have this sort of plan prepared for any
|
|
project that requires one.
|
|
#+END_NOTES
|
|
|
|
*** Chasing the Stream Processing Utopia :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Pepsi Encore
|
|
:END:
|
|
<2018-09-27 Thu 11:20-12:00>
|
|
|
|
- Architecture
|
|
- Ingestion (various sources)
|
|
- Processing (using Apache Samza)
|
|
- Complexity
|
|
- Stateless processing (filtering, /data enrichment/)
|
|
- Stateful processing (aggregation)
|
|
- Windowing
|
|
- Beam processing model
|
|
- Accuracy
|
|
- Exactly once processing
|
|
- Ingestion (de-dupe)
|
|
- Pipelined processing (Chandy Lamport)
|
|
- Storing results (idempotent writes / de-dupe)
|
|
- Scale
|
|
- Scaling ingestion
|
|
- Managed by cloud services, or yourself when self-hosting
|
|
- Scaling processing
|
|
- Challenges
|
|
- horizontal scaling
|
|
- limited to least performant component
|
|
- Typical bottlenecks
|
|
- data access
|
|
- accessing adjunct data (databases, services)
|
|
- managing temporary state
|
|
- CPU
|
|
- Any source (including batch sources, HDFS)
|
|
- Any language
|
|
- Anywhere
|
|
- Stream processing as a service
|
|
- Awesome tools
|
|
- Schema management
|
|
- Schema evolution
|
|
- ACL management
|
|
- Topic management
|
|
- Stream Processing Job Management
|
|
- Data Lineage (e.g. Apache Atlas)
|
|
- Monitoring
|
|
- Ingestion (e.g. Kafka Monitor)
|
|
- Job Progress (e.g. Burrow)
|
|
|
|
#+BEGIN_NOTES
|
|
While a good deal of the content was technology-specific, mapping out
|
|
the data flow of some of our services and seeing how they may fit in a
|
|
stream processing environment would be interesting.
|
|
#+END_NOTES
|
|
*** Thursday Lunch
|
|
:PROPERTIES:
|
|
:ADDRESS: Pappy's BBQ
|
|
:END:
|
|
<2018-09-27 Thu 12:00-13:30>
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/27 Lyft to Pappy's
|
|
Expenses:Travel:Taxis 6.46 USD
|
|
Expenses:Travel:Taxis:Tip 2.00 USD
|
|
Correl
|
|
|
|
2018/09/27 Lyft from Pappy's
|
|
Expenses:Travel:Taxis 6.48 USD
|
|
Expenses:Travel:Taxis:Tip 2.00 USD
|
|
Correl
|
|
#+END_SRC
|
|
|
|
#+BEGIN_NOTES
|
|
Really good BBQ. Enjoyed some ribs, brisket, and burnt ends.
|
|
#+END_NOTES
|
|
**** Company :noexport:
|
|
- Sophie Smithburg
|
|
- Danny Burrow
|
|
*** Rosie Pattern Language :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre - Peabody Lounge
|
|
:END:
|
|
<2018-09-27 Thu 13:30-14:10>
|
|
|
|
- Reasoning
|
|
- Regular expressions do not scale
|
|
- Combing through data to get devops insights
|
|
- libraries of named and namespaced expressions
|
|
- RPL syntax looks like a programming language
|
|
- reads/diffs like code
|
|
- executable unit tests
|
|
- macros
|
|
- RPL expressions compose
|
|
- RPL is based on parsing expression crammers, which are greedy and possessive
|
|
- makes it difficult to be accidentally inefficient
|
|
- Allows parsing recursive grammars
|
|
- Logstash (Grok) uses pattern libraries
|
|
- Can your 'grep' do this?
|
|
- =curl -s www.google.com | rosie grep -o subs net.url_common=
|
|
- Prints all matching URL sub-expressions
|
|
- =sed -n 46,49p /var/log/system.log | rosie match all.things=
|
|
- =all.things= represents a disjunction of a variety of things,
|
|
different pattern matches will be color-coded
|
|
- =head -n 1 /var/log/system.log | rosie grep -o jsonpp num.denoted_hex=
|
|
- JSON-structured output
|
|
- Formal basis
|
|
- Chomsky hierarchy
|
|
- regular - context-free - context-sensitive - recursively enumerable
|
|
- RPL (and all PEG grammars) are context-sensitive
|
|
- Shares /some/ similarities with regular expressions
|
|
- repetition (same syntax, greedy and possessive)
|
|
- character sets (simplified syntax, one name or list or range at a
|
|
time)
|
|
- Operations (look-ahead, negation, etc.) (simplified syntax)
|
|
- "choice" differs, =/= is ordered choice, possessive)
|
|
- Ships with a standard library
|
|
- Debugging
|
|
- Trace a mismatch
|
|
- =echo '17:30:4' | rosie trace time.rfc3339=
|
|
- REPL!
|
|
- Uses
|
|
- IBM uses rosie in a big-data pipeline
|
|
- Mining source code repositories
|
|
- NCSU students wrote RPL patterns to extract language features from
|
|
different languages (including bash!)
|
|
- White-listing valid input (protecting against injection attacks
|
|
per OWASP)
|
|
- Python bindings! (also, c, go), more coming
|
|
- Not very "pythonic", currently
|
|
- Resources
|
|
- http://rosie-lang.org/
|
|
- [[gitlab:rosie-pattern-language/rosie][rosie-pattern-language/rosie]]
|
|
|
|
#+BEGIN_NOTES
|
|
I really like the idea of pattern libraries and reuse (doing it with
|
|
regular expressions is /really/ hard), and the ability to specify full
|
|
grammars is very nice. I'm curious whether this could be integrated
|
|
into our validation to avoid unnecessary duplication.
|
|
#+END_NOTES
|
|
*** Architecting for Active-Active Resiliency in the Cloud :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Union Station - Grand D
|
|
:END:
|
|
<2018-09-27 Thu 14:30-15:10>
|
|
|
|
- Data and Service resiliency and durability
|
|
- Basics (Ingredients)
|
|
- Availability (normal operation time / total time)
|
|
- mean time between failure (MTBF) / mean time to repair (MTTR)
|
|
- Multiple components in series (=x= and =y=) (A = Ax * Ay)
|
|
- Availability decreases
|
|
- Multiple components in parallel (multiple services =x=) (A = 1 - (1 - Ax)^2
|
|
- Availability increases
|
|
- Basic Active Architecture
|
|
- Web -> App -> DB
|
|
- cheap, simple, fast to market
|
|
- manual intervention, low resiliency, high potential rework
|
|
- Basic Active-Passive Architecture
|
|
- Load balanced app with fail-over nodes / read-replica db
|
|
- Hot or cold standby
|
|
- decreased downtime, - backup services & data, good balance
|
|
- increased costs, complexity, potential for data loss
|
|
- Basic Active-Active Architecture #1
|
|
- Multiple instances taking traffic simultaneously
|
|
- always on, minimum data loss, zero intervention
|
|
- limited tools, most complex, most expensive
|
|
- Considerations & trade-offs (Appetizers)
|
|
- Data Replication
|
|
- Active-Active (avoid this)
|
|
- multiple master databases synchronizing
|
|
- collisions
|
|
- latency
|
|
- Active-Passive
|
|
- Multiple instances writing to a single data store
|
|
- promote a read replica on failure
|
|
- Active-Active at the service level, Active-Passive at the
|
|
data level
|
|
- Easy data replication databases (managed)
|
|
- Cloud Spanner
|
|
- Cosmos
|
|
- Aurora
|
|
- Easy data replication databases (not managed)
|
|
- Apache Cassandra
|
|
- MySQL
|
|
- Considering tenants - what are we failing over?
|
|
- /Similar to considering restart strategies in OTP supervision
|
|
trees/
|
|
- As availability and durability increases, so does complexity and cost
|
|
- Resiliency patterns (The Entrees)
|
|
- Circuit breaker (stop retrying)
|
|
- Exponential back-off (slow down retries)
|
|
- Fallback (error response or cache)
|
|
- Multiple instances in different availability zones within a
|
|
region
|
|
- Worker queues and streams decouple components
|
|
- Patterns can be combined
|
|
- Tools and services (The Desserts)
|
|
- Chaos Monkey
|
|
- Serverless
|
|
|
|
#+BEGIN_NOTES
|
|
It's important to prepare for failure, and balancing availability and
|
|
consistency is always a balancing act.
|
|
|
|
For failure, I like to map out services and their dependency as I
|
|
would an Erlang supervision tree, drawing boundaries around tightly
|
|
coupled components, and defining the failure strategy for each group.
|
|
#+END_NOTES
|
|
*** Towards Language Support for Distributed Systems :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre
|
|
:END:
|
|
<2018-09-27 Thu 15:30-16:10>
|
|
|
|
#+BEGIN_QUOTE
|
|
Research is very animated! (doesn't look like it from the outside)
|
|
#+END_QUOTE
|
|
|
|
- Language support for distributed system = ways that the programming
|
|
language/compiler can aid in the development of a distributed application
|
|
- Some areas of research
|
|
- Consistency & Programming Models
|
|
- Consistency?
|
|
- Specifically programming models that provide some kind of
|
|
consistency guarantee (e.g, what guarantees do I have for a
|
|
replica of an object/datum?)
|
|
- Sometimes weak consistency is good enough
|
|
- Conflict-free replicated data types (CRDTs)
|
|
- Lasp: A Language for Distributed, Coordination-Free
|
|
Programming
|
|
- Spry is a programming model for building applications that want
|
|
to trade-off availability and consistency at varying points in
|
|
application code to support application requirements (CA in CAP)
|
|
- Mixed consistency via "Correctables" (Incremental Consistency
|
|
Guarantees for Replicated Objects)
|
|
- Give the user different APIs depending on the sort of
|
|
consistency they're after
|
|
- =invokeWeak(operation)=
|
|
- =invokeStrong(operation)=
|
|
- =invoke(operation[, levels])=
|
|
- Mixed consistency via "MixT" ([[http://www.cs.cornell.edu/andru/papers/mixt/mixt.pdf][MixT: A Language for Mixing
|
|
Consistency in Geodistributed Transactions]])
|
|
- Answers the question of what if we want to choice from
|
|
multiple consistency options in one application
|
|
- Session Types
|
|
- Can be though of as types for communication protocols
|
|
- Many variants of session types
|
|
- Binary & Multiparty (# of parties)
|
|
- Static & Dynamic
|
|
- [[http://simonjf.com/2016/05/28/session-type-implementations.html][Introduction to session types by Simon Fowler]]
|
|
- Prototypes of session types are implemented for many languages
|
|
- Erlang :: monitored-session-erlang
|
|
- Python :: [[https://www.doc.ic.ac.uk/~rn710/spy/main.pdf][SPY]]
|
|
- Haskell :: effect-sessions
|
|
- Scala :: Ichannels
|
|
- Java :: Scribble
|
|
- Static Analysis & Verification
|
|
- Others
|
|
- Whip ([[https://people.seas.harvard.edu/~chong/pubs/icfp17-whip.pdf][Higher-Order Contracts for Microservices]])
|
|
- [[http://unisonweb.org][Unison]]
|
|
- Haskell-like language for moving computations around
|
|
- Syndicate ([[https://www2.ccs.neu.edu/racket/pubs/dissertation-garnock-jones.pdf][Conversational Concurrency]])
|
|
- ML5 ([[http://www.cs.cmu.edu/~tom7/papers/modal-types-for-mobile-code.pdf][Modal Types for Mobile Code]])
|
|
- Function-Passing ([[https://infoscience.epfl.ch/record/230304/files/fp-jfp.pdf][A Programming Model and Foundation for
|
|
Lineage-Based Distributed Computation]])
|
|
- Birds-eye view of fringe projects
|
|
- [[http://christophermeiklejohn.com/publications/hotedge-2018-containers-preprint.pdf][Verifying Interfaces Between Container-Based Components (Or... A
|
|
Type System By Any Other Name)]]
|
|
- Rejected 😅
|
|
- Statically ensuring that microservices satisfy invariants -
|
|
Adelbert Chang
|
|
- Statically ensuring that functions on replicated data are
|
|
monotone - Kevin Clancy
|
|
- Fault-Tolerant Functional Reactive Programming (Functional
|
|
Pearl)
|
|
- Resources
|
|
- http://bit.ly/dist-prog-refs
|
|
*** Keynote: Erica Joy Baker - Changing the World :session:keynote:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre
|
|
:END:
|
|
<2018-09-27 Thu 16:30-17:20>
|
|
|
|
- Passionate about talking about diversity and inclusion
|
|
- It seems that everyone wants to "change the world"
|
|
- "We did it! ... Mission accomplished, in the words of George Bush.
|
|
We changed the world"
|
|
- "We got so focused on disruption that we didn't stop to think about
|
|
who or what we were disrupting"
|
|
- Recommends watching "Halt and Catch Fire"
|
|
- "Silicon Valley is Venture Capital"
|
|
- Story of "The Scorpion and the Frog"
|
|
- The VCs are the scorpions "in case that wasn't clear".
|
|
- Steps to change the world
|
|
1. Have an idea
|
|
2. Grow your ideas slowly.
|
|
3. Get customers
|
|
4. Get some more of them
|
|
5. Keep going
|
|
6. Build what you want to build on your own terms.
|
|
- Mailchimp is a sponsor (bleh), has no VC funding
|
|
- We have to ask ourselves, "I can do this, but should I do this?"
|
|
- "How we're going to change the world is by the legacy we leave
|
|
behind."
|
|
- "We must build companies that represent everyone, and all
|
|
experiences."
|
|
- Call to action: If you work in a company that you know is doing
|
|
something harmful, that you have averted your eyes from, now is the
|
|
time to speak up and do something.
|
|
- Change the world, just make sure you're changing it for the better.
|
|
*** Dinner
|
|
:PROPERTIES:
|
|
:ADDRESS: Schlafly
|
|
:END:
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/27 Schlafly Tap Room
|
|
Expenses:Food:Dinner 19.06 USD
|
|
Correl
|
|
#+END_SRC
|
|
|
|
**** Company :noexport:
|
|
- Brooke
|
|
- Joanna Solmon
|
|
- Sophie Smithburg
|
|
- Aisha Blake
|
|
- Evelyn Ting
|
|
- Sarah Fowler
|
|
- Katherine Cox-Buday
|
|
- Paula Gearon
|
|
- Anna Liao
|
|
*** Evening
|
|
:PROPERTIES:
|
|
:ADDRESS: Union Station Hotel
|
|
:END:
|
|
|
|
#+NAME: ledger
|
|
#+BEGIN_SRC ledger
|
|
2018/09/27 Union Station Restaurant & Lounge
|
|
Expenses:Food:Drinks 11.17 USD
|
|
Correl
|
|
|
|
2018/09/27 Union Station Restaurant & Lounge
|
|
Expenses:Food:Drinks 12.78 USD
|
|
Correl
|
|
#+END_SRC
|
|
|
|
**** Company :noexport:
|
|
- Brooke
|
|
|
|
Had a great talk about everything from anime to video games to tv
|
|
shows (and animated tv shows), to software architecture and
|
|
organization / team dynamics!
|
|
** Friday
|
|
*** Friday Breakfast
|
|
<2018-09-28 Fri 08:30-09:00>
|
|
*** All the Languages Together :session:keynote:
|
|
:PROPERTIES:
|
|
:ADDRESS: Stifel Theatre
|
|
:END:
|
|
<2018-09-28 Fri 09:10-10:00>
|
|
|
|
- Different languages have different strengths
|
|
- Multi-Language software is hard
|
|
- FFI/JNI
|
|
- microservices
|
|
- Can we allow reasoning about interop at the source level?
|
|
- As opposed to running code and encountering errors / debugging
|
|
- Language specifications are incomplete - don't account for linking
|
|
- 'escape hatches' (ML/C FFI, Rust/unsafe, Java/JNI)
|
|
- We tell programmers to "be careful"
|
|
- Rethink PL Design: Principled FFIs
|
|
- Design *linking types* extensions that support safe
|
|
interoperability with other languages
|
|
- linear types (similar to rust ownership)
|
|
- pure (terminating) functions
|
|
- Only need linking type extensions to interact with
|
|
*behavior/features inexpressible* in your language
|
|
- [[https://arxiv.org/abs/1711.04559][Linking Types for Multi-Language Software: Have Your Cake and Eat It Too]]
|
|
- Reasoning about refactoring
|
|
- Let the programmer specify, do I want to only link with pure
|
|
code? Allow linking with stateful code?
|
|
- Minimal annotation burden
|
|
- Provide a default translation (e.g, a pure language defaults to
|
|
linking pure code, not stateful)
|
|
- Catching cross-language type errors
|
|
- What happens if the linked code doesn't have linking type
|
|
annotations?
|
|
- Type-preserving compilation
|
|
- Provide specifications of code invariants that can be checked at
|
|
link time
|
|
- [[https://www.cs.princeton.edu/~dpw/papers/tal-toplas.pdf][From System F to Typed Assembly Language]]
|
|
- Cross-language type errors
|
|
- Fail linking if target (compiled) types don't match
|
|
- Refactoring-Preserving Compilation
|
|
- Also known as /equivalence-preserving compilation/, /fully
|
|
abstract compilation/, and /secure compilation/
|
|
- Specifying a non-default linking type means giving up some
|
|
refactoring patterns
|
|
- Programmer gets to decide what refactorings matter to them; LT
|
|
annotations tell the compiler what linking to allow/disallow
|
|
- Linking types are the escape hatch
|
|
- Ongoing work
|
|
- Mulberry Project
|
|
- Building linking types to interop ML with Rust
|
|
- Building linking types for a pure language
|
|
- Building on top of web assembly (generating a richly typed Wasm
|
|
with state/effect encapsulation)
|
|
- What about untyped languages (Scheme, C)?
|
|
- /Untyped/ Wasm (gradual typing)
|
|
- *control* linking type
|
|
- [[http://silc.ccs.neu.edu][SILC - Secure Interoperability, Languages & Compilers]]
|
|
*** Leverage vs. Autonomy in a Large Software System :session:
|
|
:PROPERTIES:
|
|
:ADDRESS: Union Station - Grand ABC
|
|
:END:
|
|
<2018-09-28 Fri 10:20-11:00>
|
|
|
|
- Big company problems
|
|
- How do you make good use of hundreds of software engineers to
|
|
build a large system?
|
|
- How do you organize a large software system so hundreds of
|
|
engineers working on it can avoid wasting their time?
|
|
- Andy grove on org structure (High Output Management)
|
|
- functional team - increased leverage
|
|
- mission-oriented team - increased speed
|
|
- speed is the only advantage
|
|
- Conway's Law
|
|
- Software tends to look like the organization that built it
|
|
- Coplein's Law
|
|
- If your software doesn't look like your organization, you've got
|
|
trouble
|
|
- Leverage vs Autonomy
|
|
- centralized systems - increased leverage
|
|
- decentralized systems - increased autonomy
|
|
- poor leverage - waste time reinventing wheels
|
|
- poor autonomy - waste time waiting on other teams
|
|
- Costs of autonomy/decentralization
|
|
- difficult to make global improvements
|
|
- difficult to make cross-cutting changes
|
|
- uncontrolled system complexity (little consistency)
|
|
- Costs of leverage/centralization
|
|
- single points of failure / monoculture risk
|
|
- arcade knowledge (downside of knowledge specialization)
|
|
- uncontrolled system complexity (poor modularity)
|
|
- sweet spot: cluster scheduling
|
|
- increased leverage, removed repetitive operational work
|
|
- most customers want roughly the same thing
|
|
- no important loss of autonomy
|
|
- global system improvement (upgrades)
|
|
- bugs (very rarely) bring down multiple services
|
|
- sweet spot: finagle as theft service framework
|
|
- high leverage
|
|
- most customers want roughly the same thing
|
|
- no important loss of autonomy
|
|
- difficult to debug
|
|
- sour patch: monolithic service
|
|
- lots of autonomy problems (major development choke-point)
|
|
- poor sense of ownership
|
|
- very easy to cause cross-cutting bugs
|
|
- single ci/deploy pipeline
|
|
- easy to make cross-cutting changes
|
|
- good leverage
|
|
- sweet spot: service oriented architecture
|
|
- autonomy is really good
|
|
- every project needs its own ci/deployment pipeline
|
|
- difficult to cause cross-cutting bugs
|
|
- good ownership
|
|
- code is very modularized
|
|
- leverage is low
|
|
- difficult to make cross-cutting changes
|
|
- sour patch: cross-service application logic library
|
|
- where to put cross-cutting logic? attempted a shared library,
|
|
embedded in a variety of separate services
|
|
- low autonomy (lots of services to deploy with changes)
|
|
- bad ownership (which team owns this?)
|
|
- sour patch: unowned services
|
|
- consequence of autonomy
|
|
- services get lost in the shuffle between teams
|
|
- sour patch: ad-hoc service platform
|
|
- building stuff into existing services, rather than creating a new
|
|
one
|
|
- caused by an aversion to build a whole new service
|
|
- "What's the fastest way I can get this piece of work done?"
|
|
- owners of the code get a lot of leverage, but lose a lot of
|
|
autonomy
|
|
- Strato: platform for microservices
|
|
- monolithic service that hosts services inside of it
|
|
- captures some common patterns (e.g., a data store with a cache in
|
|
front of it, etc.)
|
|
- Goal is to split the systems concerns from the applications
|
|
themselves (e.g., timeouts, retries, etc.)
|
|
- Microservices within are isolated, allows modularity
|
|
- a lot easier for the code structure to match the organization
|
|
structure
|
|
- Thrift RPC
|
|
- communication between services is structurally typed
|
|
- "compatibility" relation between types makes upgrades easier
|
|
- StratoQL: a DSL for microservices
|
|
- structurally typed
|
|
- native support for thrift data
|
|
- transparent concurrency (cf. Haxl (previous Sl talk))
|
|
- large gain in leverage over separate microservices
|
|
- captures common infrastructure patterns in configuration
|
|
- autonomy for application logic owners
|
|
- many customers want roughly the same thing
|
|
- complicated interface
|
|
- centralized team a bottleneck
|
|
- difficult to debug
|
|
- downtime affects many microservices
|
|
|
|
* Friends and Acquaintances :noexport:
|
|
** Jeremy Fairbank :elm:
|
|
https://pragprog.com/book/jfelm/programming-elm
|
|
** Jesse Tomchak :elm:
|
|
** Jason Dew :elm:
|
|
** Jim Carlson :elm:
|
|
:PROPERTIES:
|
|
:twitter: epsilon2718
|
|
:END:
|
|
- Met at Tuesday elm dinner
|
|
- [[https://jxxcarlson.github.io/app/miniLatexLive/index.html][miniLatex]]
|
|
** Noah Gordon :lgbtq:elm:
|
|
** Rain :lgbtq:
|
|
** Alex Koppel :elm:lgbtq:
|
|
:PROPERTIES:
|
|
:twitter: arsduo
|
|
:END:
|
|
|
|
- Met at Wednesday lunch with LGBTQ folks
|
|
** Michael Donatz :lgbtq:
|
|
** Pernille Wangsholm :lgbtq:elm:
|
|
** Sophie Smithburg :lgbtq:
|
|
:PROPERTIES:
|
|
:twitter: tehgeekmeister
|
|
:END:
|
|
|
|
** Danny Burrow :lgbtq:
|
|
** Sara Farquharson
|
|
Awesome outfit!
|
|
** Brooke
|
|
:PROPERTIES:
|
|
:twitter: codegirlbrooke
|
|
:END:
|
|
Software engineer @ Alchemy (Cincinnati, OH)
|
|
* Ledger :noexport:
|
|
** Expenses
|
|
|
|
#+BEGIN_SRC ledger :noweb yes :cmdline register expenses --sort date --format "| %(justify(date, 20)) | %(justify(truncated(payee, 50), 50)) | %(justify(truncated(account, 40), 40)) | %(amount) |\n" :exports results :results drawer
|
|
<<ledger>>
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
:RESULTS:
|
|
| 2018/06/14 | Strange Loop 2018 | Expenses:Conference:Strange Loop | 725.00 USD |
|
|
| 2018/06/14 | Strange Loop 2018 | Expenses:Conference:elm-conf | 125.00 USD |
|
|
| 2018/07/16 | Union Station Hotel Confirmation | Expenses:Travel:Hotel | 600.00 USD |
|
|
| 2018/07/16 | Union Station Hotel Confirmation | Expenses:Travel:Hotel:Taxes | 113.58 USD |
|
|
| 2018/07/25 | Small Batch Whiskey and Fare | Expenses:Food:Lunch | 34.91 USD |
|
|
| 2018/08/06 | American Airlines | Expenses:Travel:Flights | 334.88 USD |
|
|
| 2018/08/06 | American Airlines | Expenses:Travel:Flights:Taxes | 53.52 USD |
|
|
| 2018/09/25 | American Airlines -- Checked bag | Expenses:Travel:Flights:Luggage | 25.00 USD |
|
|
| 2018/09/25 | Taxi to hotel | Expenses:Travel:Taxis | 42.14 USD |
|
|
| 2018/09/25 | Philly Pretzel Factory | Expenses:Food:Lunch | 8.49 USD |
|
|
| 2018/09/26 | The Cheese Shac | Expenses:Food:Lunch | 10.20 USD |
|
|
| 2018/09/26 | Union Station Restaurant & Lounge | Expenses:Food:Dinner | 23.60 USD |
|
|
| 2018/09/27 | Lyft to Pappy's | Expenses:Travel:Taxis | 6.46 USD |
|
|
| 2018/09/27 | Lyft to Pappy's | Expenses:Travel:Taxis:Tip | 2.00 USD |
|
|
| 2018/09/27 | Lyft from Pappy's | Expenses:Travel:Taxis | 6.48 USD |
|
|
| 2018/09/27 | Lyft from Pappy's | Expenses:Travel:Taxis:Tip | 2.00 USD |
|
|
| 2018/09/27 | Schlafly Tap Room | Expenses:Food:Dinner | 19.06 USD |
|
|
| 2018/09/27 | Union Station Restaurant & Lounge | Expenses:Food:Drinks | 11.17 USD |
|
|
| 2018/09/27 | Union Station Restaurant & Lounge | Expenses:Food:Drinks | 12.78 USD |
|
|
| 2018/09/28 | Taxi from hotel | Expenses:Travel:Taxis | 46.64 USD |
|
|
| 2018/09/28 | American Airlines -- Checked bag | Expenses:Travel:Flights:Luggage | 25.00 USD |
|
|
| 2018/09/28 | Philadelphia Parking Authority | Expenses:Travel:Parking | 44.00 USD |
|
|
:END:
|
|
|
|
** Balances
|
|
#+BEGIN_SRC ledger :noweb yes :cmdline bal --no-total -F "| %(total) | %(account) |\n" :exports results :results drawer
|
|
<<ledger>>
|
|
#+END_SRC
|
|
|
|
#+RESULTS:
|
|
:RESULTS:
|
|
| -1951.98 USD | AWeber |
|
|
| -319.93 USD | Correl |
|
|
| 2271.91 USD | Expenses |
|
|
| 850.00 USD | Expenses:Conference |
|
|
| 725.00 USD | Expenses:Conference:Strange Loop |
|
|
| 125.00 USD | Expenses:Conference:elm-conf |
|
|
| 120.21 USD | Expenses:Food |
|
|
| 42.66 USD | Expenses:Food:Dinner |
|
|
| 23.95 USD | Expenses:Food:Drinks |
|
|
| 53.60 USD | Expenses:Food:Lunch |
|
|
| 1301.70 USD | Expenses:Travel |
|
|
| 438.40 USD | Expenses:Travel:Flights |
|
|
| 50.00 USD | Expenses:Travel:Flights:Luggage |
|
|
| 53.52 USD | Expenses:Travel:Flights:Taxes |
|
|
| 713.58 USD | Expenses:Travel:Hotel |
|
|
| 113.58 USD | Expenses:Travel:Hotel:Taxes |
|
|
| 44.00 USD | Expenses:Travel:Parking |
|
|
| 105.72 USD | Expenses:Travel:Taxis |
|
|
| 4.00 USD | Expenses:Travel:Taxis:Tip |
|
|
:END:
|