عجفت الغور

emnlp 2021

grad school, nlp

Keynote at CONLL

  • harmonics in linguistics
  • what does artificial language learning look like?
  • english has a harmonic pattern with VP and PP, while chinese does not
    • some evidence that people believe the verb phrase order generalizing it from one phrase type to another
  • how to people generalize to noisy data, do people favory harmony
    • no matter what, across prior patterns, people tend to favor non-harmonic
      • child learners tend to favor harmonic patterns
  • hypothesis: are the frequent orders of free-order languages homomorphic to the underlying structure?
    • can learners infer a homomorphic order?
  • even when language is non-homomorphic, they tend to favor homomorphic orders in artificial langauges
  • which attributes of an object are closely associated with it?
  • learners do not simply reproduce their output, their failure to correctly reproduce shows particular heuristics
    • what is the relationship between imperfect learning and biases?
  • noun classes/genders
    • classes are not purely idiosyncratic
  • typology - semantic cues are ubiquitous
    • reflects salient, often highly reliable features
  • children favor phonology
    • possibly because they don’t know the meanings of nouns?
    • inherent bias for noun-internal cues
  • showing core properties for human learners, phonological vs semantics
    • we’re sensitive to features of the input
      • is the information entropy of a dataset reducable to the information entropy required to generate the dataset?

Creole Modeling

  • socioally dominant groups
  • creole is born from many different languages
  • distributionally robust optimization
    • does the distribution of languages within the creole actually convey information?
  • can DRO help us create better LMs for creoles?
  • creole-only datasets
    • fasttext, using confidence scores
  • arabic/chinese based creole -> how do the confidence scores for fasttext work for latin script
  • why does DRO not work?

LM’s and Telicity

  • telecity - whether an event has an inherent endpoint or boundary
  • typically selecting of “temporal adverbial phrases”
  • the object np is important for determining telecity

Geometric Spaces

  • is linear information in low-dimensional subspaces captured in LM’s?
  • augmenting the probe model with low-rank linear projection
  • are the axis of alignment in the low-rank that map to specific neurons?
  • argues that LMs rely on low-dim encodings
  • inlp - changes the representation maximally?

German Plural Generalization

  • predicting the plural from the gender and the noun transformation
  • modeling othrographic transformations
  • suffixes of majority classes are overly generalized
  • interesting section on intervention, intervention caused the model to start to make spelling mistakes
  • prediction of plurals for arabic?
    • if we train a NN to predict plurals in arabic, does that tell us anything about how people learn plurals in arabic?

Can languages models encode color?

  • LM’s can capture relational knowledge, can they capture color?
  • LM’s can also capture and inferred latent relationships
  • color is interesting because it’s very similar to usage of color words in cognitive science
  • color terms hold linguistic significance
  • topological spaces of colors are somewhat projected into the color space
  • color terms represent based on simple co-occurence already demonstrate correspondance, LM’s are aligned more closely