All Articles

Coursera μ‹€ν—˜ μˆ˜μ—…

올 가을뢀터 Coursera 에 Interaction Designμ΄λΌλŠ” specialization 이 μƒκ²ΌλŠ”λ° κ·Έ 쀑 Designing, Running, and Analyzing Experiments νŒŒνŠΈλŠ” μ–΄λ””κ°€μ„œ 배우기 νž˜λ“  λ‚΄μš©μ΄λΌκ³  κ΅μˆ˜λ‹˜μ΄ λ“£κΈ°λ₯Ό κ°•λ ₯ μΆ”μ²œν•˜μ…¨μ§€λ§Œ λ‹Ήμ‹œμ—λŠ” 듣지 μ•Šμ•˜λ‹€(μ£„μ†‘ν•©λ‹ˆλ‹€β€¦). κ·ΈλŸ¬λ‹€ μš”μ¦˜ 학기와 겨울 λ°©ν•™ 사이 μ•½κ°„μ˜ ν‹ˆμ΄ 생겨 9 주짜리 μ½”μŠ€μ§€λ§Œ μ§‘μ€‘μ μœΌλ‘œ λ“£κ³ , 그쀑에 μ’€ 남길 λ§Œν•œ λ‚΄μš©μ„ 남겨보렀고 ν•œλ‹€.


Week1μ—λŠ” μ‹€ν—˜μ΄ μ™œ ν•„μš”ν•œμ§€μ™€ μ‹€ν—˜ λ””μžμΈμ— λŒ€ν•œ 큰 그림이 λ‚˜μ˜¨λ‹€.

μ‹€ν—˜μ΄ μ™œ ν•„μš”ν•œκ°€

μš°λ¦¬κ°€ μ‹€ν—˜μ„ ν•΄μ•Όν•˜λŠ” μ΄μœ λŠ” λ””μžμΈμ„ 수치적으둜 μ΄ν•΄ν•˜κ³ , 증λͺ…ν•΄μ„œ κ²°κ΅­ λ””μžμΈμ„ κ°œμ„ ν•˜λŠ”λ° λͺ©μ μ΄ μžˆλ‹€. κ·Έ μΈ‘μ •μ˜ λŒ€μƒμ€ Performanceκ°€ 될 μˆ˜λ„ 있고, Preferenceκ°€ 될 μˆ˜λ„ μžˆλ‹€. λ˜ν•œ μ‹€ν—˜μ€ Exploratory ν•  μˆ˜λ„ 있고, μ•„λ‹ˆλ©΄ μ’€ 더 ꡬ체적인 Hypothesis&λ₯Ό κ²€μ¦ν•˜λŠ” λ°©μ‹μœΌλ‘œ 진행될 μˆ˜λ„ μžˆλ‹€. κ°•μ˜μ—μ„œλŠ” μ›Ήμ‚¬μ΄νŠΈμ˜ λ””μžμΈμ„ κ°œνŽΈν–ˆμ„ λ•Œ μ‚¬μš©μžμ˜ μ„ ν˜Έλ„λ₯Ό μ•Œμ•„λ³΄κΈ° μœ„ν•œ A/B Test λ₯Ό 예둜 λ“€μ—ˆλŠ”λ°, 사싀 이 경우 μΈ‘μ • 였차 λ•Œλ¬Έμ— 두 개의 같은 λ””μžμΈμ„ 놓고 μ‹€ν—˜ν•΄λ„ 차이가 λ‚˜νƒ€λ‚  수 μžˆλ‹€. (이런 효과λ₯Ό μΈ‘μ •ν•˜κΈ° μœ„ν•œ μ‹€ν—˜μ„ A/A Testing이라고 ν•œλ‹€. Ron Kohavi λ‹˜μ˜ μ €μˆ λ“€ 쀑에 이 μͺ½μœΌλ‘œ μž¬λ°ŒλŠ” λ‚΄μš©μ΄ λ§Žλ‹€)

μ‹€ν—˜ λ””μžμΈμ˜ 큰 κ·Έλ¦Ό

κ°•μ—°μžκ°€ μŠ¬λΌμ΄λ“œ ν•˜λ‚˜ 없이 μ•žμ—μ„œ μ­‰ κ°•μ˜ν•΄μ„œ (일단 ν€΄μ¦ˆλ₯Ό ν’€κΈ° μœ„ν•΄) Bullet Point 둜 μ •λ¦¬ν–ˆλ‹€.

Statistical significance vs. Practical significance

  • Statistical significance: mathematical probability that a relationship btw. two or more variable exists

    • Eg. Testing preference: We need hypothesis and test statistic. It returns confidence
  • Practical significance: actual difference it is estimating will affect a decision to be made

4 Major considerations for every experiment

  1. Participants

    • Sampling: how do they become part of our study?

      • purpose: Make composition of sample similar to real
    • Probability sampling: consisting randomness
    • Non-probability sampling

      • Convenience sampling: At researcher’s convenience
      • Snowball sampling: Random first sample -> he/she introduce next sample
    • Inclusion / Exclusion criteria

      • If we experiment users’s preference between old & new design of web site, we may exclude people who are already familiar with old design
  2. Apparatus

    • What do we need in terms of equipment, space, and other resources
    • Remote(same time, different place) / online study(different time and place)
    • Do we build something? how to capture data? (human? video? log? )
  3. Procedure

    • What do they actually go through as they come into the study
    • There might be some side effect (fatigue effect, learning effect)
    • Informed consent is very important!: something similar to IRB

      • Don’t forget you’re in charge of their welfare during that time
      • Is there any potential risk? Space should be accessible for blind, deaf, etc…
      • At the end of study, you have to debrief your study to participants
    • You may have seperate MC for proceeding experiment, while other one recording data.
  4. Design and analysis

    • What do they do on each site

P.S

  • Week2 λΆ€ν„°λŠ” 자주 μ“°μ΄λŠ” 뢄석을 R 둜 ν•˜λ‚˜μ”© ν•΄λ³Έλ‹€κ³  ν•˜λŠ”λ°, μ–Έμ  κ°€ ν”„λ‘œλ•νŠΈμ— 넣을 λ•Œλ₯Ό μƒκ°ν•΄μ„œ Python 기반의 툴둜 λ”°λΌκ°ˆ μˆ˜λŠ” μ—†μ„κΉŒ ꢁ리쀑이닀. 이건 듀어보고 μ—…λ°μ΄νŠΈ.
  • κ°•μ˜λ₯Ό 쑰금 λ“€μ–΄λ³΄λ‹ˆ μ–΄λ–€ μ–΄λ–€ κ²½μš°μ—λŠ” X 뢄석을 λŒλ¦¬μ„Έμš”. X 뢄석은 μ΄λŸ°κ±°μ—μš”. μ‹μœΌλ‘œ κ°•μ˜κ°€ ν˜λŸ¬κ°„λ‹€. R 둜 ν•˜λŠ” 일이라고 ν•΄λ΄€μž μ–΄μ°¨ν”Ό 데이터λ₯Ό μ²˜λ¦¬ν•΄μ„œ 블둝킹 μ—°μ‚° ν•˜λ‚˜μ— λ„£λŠ”κ±°κ³ , λ³„λ‘œ 무거울 것도 μ—†μ–΄μ„œ Python 기반으둜 ν•˜λŠ” 건 ν•„μš”ν•  λ•Œ 해도 μΆ©λΆ„νžˆ ν•˜κ² λ‹€ μ‹Άμ—ˆλ‹€.
Published 19 Dec 2016

If I keep marking the dots, someday they will πŸ”—πŸ”—
Hyeungshik Jung on Twitter