Keyboard shortcuts

/ or ⌘/Ctrl K
Find a note
j / k
Next / previous section or linked note
h / l
Collapse or go to parent / expand or enter
e or Alt-click
Read a linked note here
o
Open focused note on its own
g g / G
First / last section or linked note
g h / g a
Home / all notes
g b / g t
Backlinks / table of contents
t
Cycle System, Light, Dark
? / Esc
Show / close this reference

Search: ↑/↓ or Ctrl N/P, Enter to open. Shortcuts pause while typing.

emnlp 2021 [44a35b91]

grad school, nlp

1. Keynote at CONLL

  • harmonics in linguistics
  • what does artificial language learning look like?

  • english has a harmonic pattern with VP and PP, while chinese does not

    • some evidence that people believe the verb phrase order generalizing it from one phrase type to another

  • how to people generalize to noisy data, do people favory harmony

    • no matter what, across prior patterns, people tend to favor non-harmonic

      • child learners tend to favor harmonic patterns

  • hypothesis: are the frequent orders of free-order languages homomorphic to the underlying structure?

    • can learners infer a homomorphic order?
  • even when language is non-homomorphic, they tend to favor homomorphic orders in artificial langauges
  • which attributes of an object are closely associated with it?

  • learners do not simply reproduce their output, their failure to correctly reproduce shows particular heuristics

    • what is the relationship between imperfect learning and biases?

  • noun classes/genders

    • classes are not purely idiosyncratic

  • typology - semantic cues are ubiquitous

    • reflects salient, often highly reliable features

  • children favor phonology

    • possibly because they don't know the meanings of nouns?
    • inherent bias for noun-internal cues

  • showing core properties for human learners, phonological vs semantics

    • we're sensitive to features of the input

      • is the information entropy of a dataset reducable to the information entropy required to generate the dataset?

1.1. Creole Modeling

  • socioally dominant groups
  • creole is born from many different languages

  • distributionally robust optimization

    • does the distribution of languages within the creole actually convey information?
  • can DRO help us create better LMs for creoles?

  • creole-only datasets

    • fasttext, using confidence scores
  • arabic/chinese based creole -> how do the confidence scores for fasttext work for latin script
  • why does DRO not work?

1.2. LM's and Telicity

  • telecity - whether an event has an inherent endpoint or boundary
  • typically selecting of "temporal adverbial phrases"
  • the object np is important for determining telecity

1.3. Geometric Spaces

  • is linear information in low-dimensional subspaces captured in LM's?
  • augmenting the probe model with low-rank linear projection
  • are the axis of alignment in the low-rank that map to specific neurons?
  • argues that LMs rely on low-dim encodings
  • inlp - changes the representation maximally?

1.4. German Plural Generalization

  • predicting the plural from the gender and the noun transformation
  • modeling othrographic transformations
  • suffixes of majority classes are overly generalized
  • interesting section on intervention, intervention caused the model to start to make spelling mistakes

  • prediction of plurals for arabic?

    • if we train a NN to predict plurals in arabic, does that tell us anything about how people learn plurals in arabic?

1.5. Can languages models encode color?

  • LM's can capture relational knowledge, can they capture color?
  • LM's can also capture and inferred latent relationships
  • color is interesting because it's very similar to usage of color words in cognitive science
  • color terms hold linguistic significance
  • topological spaces of colors are somewhat projected into the color space
  • color terms represent based on simple co-occurence already demonstrate correspondance, LM's are aligned more closely