LLMs for software development can go way beyond Github Copilot. In this episode, we talk about how these models could change the workflow for existing developers, but more importantly how they could change the very idea of what software is.

Links/Resources:

https://simonwillison.net/2023/Mar/11/llama/

https://github.com/RootbeerComputer/backend-GPT

https://lukebechtel.com/blog/gpt4-generating-code

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Dependent types are a more expressive type system in programming languages used to catch a larger class of errors at compile time. What are would be typically assertions at runtime can now be caught at compile time.

Show notes:

• Proposition as Types
• Parse, Don’t Validation
• “Scala vs Idris: Dependent types, now and in the future”

Resources:

• http://www.e-pig.org/downloads/ydtm.pdf
• https://gist.github.com/Hirrolot/27e6b02a051df333811a23b97c375196
• Proof Theory Impressionism: Blurring the Curry-Howard Line
• Type Systems - The Good, Bad and Ugly
• Dependent types for practical use
• Idris: Practical Dependent Types with Practical Examples
• Making Illegal States unrepresentable
• Can types replace validation
  https://www.cs.ox.ac.uk/ralf.hinze/WG2.8/26/slides/xavier.pdf

Activity Pub is a decentralized social networking protocol. It allows different web applications to interact so that their users can share information, even if the websites or applications are running different software.

00:00 S04E07 Activitypub
02:13 What is Activity Pub?
15:02 Interoperability
33:00 A New Kind of Social
43:53 Providing Distribution
53:03 The Use Cases
01:07:41 Ideology
01:14:25 The Wild West
01:24:07 Deconstructed Content
01:34:52 Taking a Position

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Links/Resources:

• https://tinysubversions.com/notes/reading-activitypub/
• https://flak.tedunangst.com/post/ActivityPub-as-it-has-been-understood
• https://web.archive.org/web/20201124231343/https://cjslep.com/c/blog/an-activitypub-philosophy
• https://overengineer.dev/blog/2018/02/01/activitypub-one-protocol-to-rule-them-all.html
• https://overengineer.dev/blog/2019/01/13/activitypub-final-thoughts-one-year-later.html
• https://www.jeremydormitzer.com/blog/what-is-activitypub.html
• https://tinysubversions.com/notes/decentralized-social-networks/
• https://kyefox.com/post/707900440336039936/activitypub-could-be-the-future
• https://ariadne.space/2019/01/07/activitypub-the-worse-is-better-approach-to-federated-social-networking/
• https://activitypub.rocks/
• https://overengineer.dev/blog/2019/01/13/activitypub-final-thoughts-one-year-later.html
• https://socialhub.activitypub.rocks/t/introduction-to-activitypub/508
• https://github.com/joyeusenoelle/GuideToMastodon
• https://victoria.dev/blog/why-pixelfed-wont-save-us-from-instagram/
• https://web.archive.org/web/20190713233100/https://blog.dereferenced.org/federation-what-flows-where-and-why
• https://web.archive.org/web/20190108060531/https://blog.dereferenced.org/activitypub-the-worse-is-better-approach-to-federated-social-networking
• https://www.eugenewei.com/blog/2021/9/29/and-you-will-know-us-by-the-company-we-keep
• https://twitter.com/AIMindFlow

APL is an array based programming language developed by Kenneth Iverson in the 1960s. Its central data type is the multi-dimensional array and hence it's very useful for workloads involving a lot of matrix math. APL predominantly uses symbols and it leverages consistent composability and execution rules to enable it as a notation as a tool for thought.

Links/Resources:
* https://mathspp.com/blog/why-apl-is-a-language-worth-knowing
* https://www.quora.com/What-made-APL-programming-so-revolutionary?share=1
* https://tryapl.org/
* A Personal View of APL
* Notation as a Tool for Thought
* Heaviside Operator Calculus
* Conway’s Game of Life in APL
* Maxwell’s Equations: From 20 to 4
* Alan Kay’s answer to “What made APL so revolutionary?”
* Testimonies
* https://news.ycombinator.com/item?id=27463149
* https://news.ycombinator.com/item?id=27464005
* https://news.ycombinator.com/item?id=27465512
* https://github.com/razetime/ngn-k-tutorial/blob/main/01-intro.md
00 - Objectives and Prerequisites | "Learn APL with Neural Networks"
https://dl.acm.org/doi/pdf/10.1145/3315454.3329960

Chapters:
[00:00:00] Intros
[00:03:36] What is APL?
[00:05:42] Yes, it uses weird symbols
[00:06:51] A language should change your thinking
[00:13:08] Notation as a tool of thought
[00:20:33] Connections to functional programming
[00:28:23] What it feels like to program in APL
[00:33:10] APL and IBM = big bucks
[00:35:20] The echos of APL in modern data science tools
[00:43:52] APL for big data pipelines
[00:48:56] APL for Machine learning
[00:54:59] APL for JSON manipulation
[01:03:18] APL as a spreadsheet language
[01:16:11] Superhuman programming: APL and LLMs
[01:26:59] Making APL more explainable
[01:38:18] Outro

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

io_uring is a new asynchronous, API to enable fast and efficient system calls in the Linux kernel. It promises faster user-land programs that heavily use file system IO on Linux, for all applications, not just databases.

00:00 S04 E05 io_uring
02:55 What is io_uring?
05:59 What's wrong with existing async interfaces?
17:25 Copying is slow
19:05 Two ring buffers
28:07 What would the effect for the end user be?
34:50 Different operating modes
37:27 C10K for Filesystem IO
43:04 Speed is simplicity
47:07 Deconstructed databases
01:02:12 Effect systems
01:06:26 System functionality in programming language
01:10:55 OS as Actor Model
01:16:08 The Kernel is Userland

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: [https://technium.transistor.fm/](https://technium.transistor.fm/)
SPOTIFY: [https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7](https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7)
APPLE PODCASTS: [https://podcasts.apple.com/us/podcast/the-technium/id1608747545](https://podcasts.apple.com/us/podcast/the-technium/id1608747545)

Xanadu was the first hypertext project founded in 1960 by Ted Nelson. It aims to facilitate a type of media called hypermedia, which is non-sequential writing in which the reader can choose their own path through an electronic document.

Links/Resources:
http://www.rheingold.com/texts/tft/14.html
https://en.wikipedia.org/wiki/Project_Xanadu
https://mimix.io/en/blog/xanadu
https://sentido-labs.com/en/library/201904240732/Xanadu%20Hypertext%20Documents.html
https://www.notion.so/blog/ted-nelson
https://www.wired.com/1995/06/xanadu/
https://xanadu.com.au/ararat
https://www.youtube.com/watch?v=-_-5cGEU9S0
https://www.youtube.com/watch?v=hMKy52Intac
https://www.youtube.com/watch?v=1gPM3GqjMR4
https://www.youtube.com/watch?v=hGKbRcvIZT8
https://www.youtube.com/watch?v=qyzgoeeloJA
https://www.youtube.com/watch?v=_xYwgJW7T8o
https://jasoncrawford.org/the-lessons-of-xanadu
https://blockprotocol.org/
https://github.com/subconsciousnetwork/noosphere/blob/main/design/explainer.md
https://maggieappleton.com/xanadu-patterns
https://aaronzlewis.com/blog/2019/05/01/spreading-threading/
https://www.zombo.com/
https://stratechery.com/concept/aggregation-theory/
https://maggieappleton.com/tools-for-thought
https://cdixon.org/2015/01/31/come-for-the-tool-stay-for-the-network

Chapters:
[00:00:00] Intros
[00:03:22] What is Xanadu?
[00:15:49] Transclusion and Bidirectionality
[00:26:32] Versioning
[00:29:47] Vision divorced from implementation
[00:35:13] Baked in Payments
[00:46:15] Hypermedia as Envisioned
[00:56:20] Tiktok as Hypermedia
[01:01:52] Alternative business model for the web
[01:16:19] Failure to Launch
[01:26:15] Linearization as a forge
[01:31:51] Success of Xanadu's Vision
[01:37:04] Passing the torch

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

ChatGPT is a language model by OpenAI which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

00:00 S04E03 ChatGPT Part 2
02:30 Expanding Humanity's Limits
11:33 Tool-assisted speedrun of Science
16:18 As a Civic Moderator
24:32 Social Upheaval
28:40 Platform Risk
49:51 Billion Dollar Solo Founder
58:35 Running out of Data
01:03:22 In other industries
01:24:52 Limitations
01:31:50 Build something that gets better as AI gets better

Links:

https://chat.openai.com/

https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulator#Simulators

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

https://xyproblem.info/

https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE:

https://technium.transistor.fm/

SPOTIFY:

https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

APPLE PODCASTS:

https://podcasts.apple.com/us/podcast/the-technium/id1608747545

ChatGPT is a language model by OpenAI which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

00:00 S04E03 ChatGPT Part 1
02:58 What is ChatGPT?
06:17 Lineage of ChatGPT
10:01 General Reasoning from Training on Code
25:13 Reinforcement Learning from Human Feedback
29:37 Dumb compute, stacked high
42:00 Talking to an Alien Intelligence
45:14 What's it good at and bad at?
56:05 The XY problem
01:03:36 GPT in Education
01:13:26 Eating High Status Jobs, Fulfilling Latent Demand

Links:
- https://chat.openai.com/
- https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulator#Simulators
- http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- https://xyproblem.info/

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545