AI Canon

AI Canon

A16Z.COM
514
219
nihit-desai
5d

Comments

@negamax 5d
I am sorry but I am not a believer in a16z anything after their massive crypto token scams and wealth extraction. We all need to move away from all these companies who continue to bloat in private and then have a big pay day as a public company.
@jhp123 5d
If you click the domain on this submission, you'll see loads of articles from a16z on the topic of generative AI.

Click back a couple years and you'll find this page: https://news.ycombinator.com/from?site=a16z.com&next=2981684... with submissions like "DAOs, a Canon" https://news.ycombinator.com/item?id=29440901

@ryanSrich 5d
Looking at these comments, I can't think of another VC that has burned as much goodwill among technical people as a16z has. Don't get me wrong, it's well deserved, but it's just surprising how universal it seems to be (at least in this thread).
@xpe 5d
> Research in artificial intelligence is increasing at an exponential rate.

Probably in the blundering sense of "exponential", meaning a lot. But what are some specific numbers? (such as publications)

@rdli 5d
I was an early member of the CNCF community (circa 2016), and at the time I thought "wow things are moving quickly." Lots of different tech was being introduced to solve similar problems -- I distinctly remember multiple ways of templating K8S YAML :-).

Now that I'm spending time learning AI, it feels the same -- but the innovation pace feels at least 10x faster than the evolution of the cloud native ecosystem.

At this point, there's a reasonable degree of convergence around the core abstractions you should start with in the cloud-native world, and an article written today on this would probably be fine a year from now. I doubt this is the case in AI.

(Caveat: I've only been learning about the space for about 4 weeks, so maybe it's just me!)

@sharemywin 5d
Build AI or just invest in chip makers?

https://a16z.com/2023/01/19/who-owns-the-generative-ai-platf...

Over the last year, we’ve met with dozens of startup founders and operators in large companies who deal directly with generative AI. We’ve observed that infrastructure vendors are likely the biggest winners in this market so far, capturing the majority of dollars flowing through the stack. Application companies are growing topline revenues very quickly but often struggle with retention, product differentiation, and gross margins. And most model providers, though responsible for the very existence of this market, haven’t yet achieved large commercial scale.

In other words, the companies creating the most value — i.e. training generative AI models and applying them in new apps — haven’t captured most of it

@davidhunter 5d
I hope Tyler Cowen can ask Marc Andreessen how AI works so that we can all learn something from the master
@dpflan 5d
Looking at the authors, was this created by experts in AI? Is it sufficient to truly be a `canon`?
@oh_sigh 5d
A16Z: Friendship ended with Blockchain. Now AI is my best friend.

What's the last investment A16Z was actually ahead of the curve on? I guess it isn't important, since from their position, they don't rely on being ahead of the curve in order to make good investments, they make their investments good through their network and funding abilities.

@WoahNoun 5d
It's just a list of links with no real substance. Don't they have some crypto scams to attend to?
@alanpage 5d
And why should we trust their judgement about anything, after they put money into Adam Neuman's new company (after the WeWork debacle)?

CNBC: https://www.cnbc.com/2022/08/15/a16z-to-invest-in-adam-neuma...

@lwneal 5d
This is a fine list, but it only covers a specific type of generative AI. Any set of resources about AI in general has to at least include the truly canonical Norvig & Russel textbook [1].

Probably also canonical are Goodfellow's Deep Learning [2], Koller & Friedman's PGMs [3], the Krizhevsky ImageNet paper [4], the original GAN [5], and arguably also the AlphaGo paper [6] and the Atari DQN paper [7].

[1] https://aima.cs.berkeley.edu/

[2] https://www.deeplearningbook.org/

[3] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...

[4] https://proceedings.neurips.cc/paper_files/paper/2012/file/c...

[5] https://arxiv.org/abs/1406.2661

[6] https://www.nature.com/articles/nature16961

[7] https://www.nature.com/articles/nature14236

@boringg 5d
Anyone else feel like we've seen peak A16z at this point?
@zeroxfe 5d
> Andrej Karpathy was one of the first to clearly explain (in 2017!) why the new AI wave really matters.

Geoff Hilton had been saying this well before 2017. I remember his talks at Google ~2013ish.

@TradingPlaces 5d
Came for everyone roasting a16z. Was not disappointed.
@uptownfunk 5d
Wow, why so much hate against a16z. There's a really funny clip about Marc on the Rogan podcast where he is like "I have to come on Rogan, there's so much clout" or something to that effect. Rogan was immediately like "igghh".
@whywhywhywhy 5d
Getting whiplash from the 90 degree handbrake turn the crypto grifters have taken into being AI grifters.
@mirekrusin 5d
Why everybody (including this a16z dude) underestimates/not mentions:

1. quality of input data - for language models that are currently setup to be force-fed with any incoming data instead of real training (see 2.) this is the greatest gain you can get for your money - models can't distinguish between truth and nonsense, they're forced to follow training data auto-completion regardless of how stupid or sane it is

2. evaluation of input data by the model itself - self evaluating what is nonsense during training and what makes sense/is worthy of learning - based on so far gathered knowledge, dealing with biases in this area etc.

Current training methods equate things like first order logic with any kind of nonsense - having on its defense only quantity, not quality.

But there are many widely repeated things that are plainly wrong. Simplifying this thought - if there weren't, there would be no further progress in human kind. We constantly reexamine assumptions and come up with new theories leaving solid axioms untouched - why not teach this approach/hardcode it into LLMs?

Those two aspects seem to be problems with large gains, yet nobody seems to be discussing them.

Align training towards common/self sense, good/own judgement, not unconditional alignment towards input data.

If fine-tuning works, why not start training with first principles - dictionary, logic, base theories like sets, categories, encyclopedia of facts (omitting historic facts which are irrelevant at this stage) etc. - taking snapshots at each stage so others can fork their own training trees. Maybe even stop calling fine-tuning fine-tuning, just learning stages. Let researchers play with paths on those trees and evaluate them to find something more optimal, find optimal network sizes for each step, allow models to gradually grow in size etc.

To rephrase it a bit - we're saying that base models learned on large data work well when fine tuned - why not base models trained on first principles can continue to be trained on concepts that depend on previously learned first principles recursively efficiently - did anybody try?

@mark_l_watson 4d
Well, that is a good list. I would guess that I have only previously read the content from about 15% of the links, oh well!

Like everyone else, starting about a year and a half ago I have found it really difficult to stay up to date.

I try to dive deep on a narrow topic for several months and then move on.

I am just wrapping up a dive into GPT+LangChain+LlamaIndex applications. I am now preparing to drop most follows on social media for GPT+LangChain+LlamaIndex and try to find good people and companies to follow for LLM+Knowledge Graphs (something I tried 3 years ago, but the field was too new).

I find that when I want to dive into something new the best starting point is finding the right people who post links to the best new papers, etc.

@SilverBirch 4d
From the people who bought you web3. Look where the crowd is going, run to the front and shout "Follow me!".