User:Daniel Gregoire/Video Series for Statistics and Mathematics for Data Science and Data Analytics

I've been working my way through a video course series by Nikolai Schuler entitled Statistics and Mathematics for Data Science and Data Analytics, which I have access to through my ACM membership.

Here is a working session I did in J for parts of the Probability Theory unit.

Probability Theory

Video Link (Paywall)

Addition Rule:

$P(E_{a}\cup E_{b})=P(E_{a})+P(E_{b})-P(E_{a}\cap E_{b})$

NB. Probabilities with 2 dice

]P1=.All=.36x NB. 6*6
PFn=.%&P1
sum2=.+/^:2 NB. Sum all items in rank-2 array
die=.1+i.6
die +table die NB. Visualize the 36 combos
]ps=.die ,"0/ die NB. Use <@, to see this as in APL

]P2=.PFn sum2 9< die +/ die  NB. 1r6
]P3=.PFn sum2 (+./"1) 4<ps   NB. 5r9
]P4=.PFn sum2 (*./"1) 3>ps   NB. 1r9

NB. Probabilities as Car Salesman
PL=.0.2
PW=.0.4
PLW=.0.15
]P5=.0.2 + 0.4 - 0.15 NB. 0.45

NB. Probabilities as retail analyst
NB. 0.6 * Purchases
NB. 0.4 * Purchases had no clothing, no food.
PC=.0.3
PF=.0.5
PCorF=.0.6
NB. PCorF = P(Clothing) + P(Food) - P(Clothing \cup Food)
NB. 0.6   = 0.3         + 0.5     - x
NB. 0.6   = 0.8 - x
NB. x     = 0.8 - 0.6
]P6 =. (PC + PF) - PCorF NB. 0.2 or 20%

Multiplication Rule:

$P(E_{a}\cap E_{b})=P(E_{a})\cdot P(E_{b}|E_{a})$

NB. Two queens, without putting one back
]PQ1=.4%52x
]PQ2=.3%51x
]PTwoQueens=.PQ1 * PQ2 NB. 1r221 or 0.00452489 or 0.452% chance

NB. Container with:
NB. 7 black balls
NB. 3 white balls
NB. Draws 1st, keeps it
NB. Draws 2nd, keeps it
NB. Probability that both are black?
]PB1=.7%10x
]PB2=.6%9x
]PBothBlack=.PB1*PB2 NB. 7r15 or 46.6667% probability

Expected Value:

$E(X)=(x_{1}\cdot p_{1})+(x_{2}\cdot p_{2})+\ldots +(x_{n}\cdot p_{n})$

NB. Profit of $50 per laptop
NB. Loss of $400 per returned
NB. Probability of 4% that it's returned

]LX1=.50
]LP1=.0.96
]LX2=.-400
]LP2=.0.04
]LEX=.(LX1*LP1) + (LX2*LP2) NB. $32

NB. Wheel of Fortune event at a Restaurant

NB. 9 sections on wheel:
NB.   - 1 area with 20 euro discount
NB.   - 3 areas with 10 euro discount
NB.   - 3 areas with 5 euro discount
NB.   - 2 areas with 0 euro discount
NB. Q: Expected value of discount per spin of this wheel?

NB. Written out by hand
]REX=.(20 * (1%9x)) + (10 * (3%9x)) + (5 * (3%9x)) + (2 * (0%9x)) NB. 65r9 or 7.22 euros
NB. More array programming
RD=.2 4$20 10 5 0 1 3 3 2   NB. Data: discounts + number of areas with them
n=.0x + +/1{RD              NB. Data: Total number of areas, extended precision
]RTab=.(0{RD) ,: n%~1{RD    NB. Analysis: Probabilities in second row
]REX=.+/*/RTab              NB. Analysis: Expected discount value per spin

NB. Which leads to the beautiful definition of expected value
NB. given a 2-by-N matrix of outcomes and probabilities:
ExFn=.+/@:*/
ExFn RTab NB. 65r9