Mathematica - intro Calc part 1

Posted on April 4, 2021

Tags: mathematica

AArrow[{p_, q_}, label_:""] := {Inset[Style[label,Purple],Midpoint[{p,q}]], Arrow[{p,q}]};
vec2[p_] := Graphics@AArrow[{ConstantArray[0,Length[p]],p}];
vec2[p_,q_] := Which[  ArrayQ[q] == False, Graphics@AArrow[{ConstantArray[0,Length[p]],p},q],
                       ArrayQ[q] == True, Graphics@AArrow[{p,q},""] ]
vec2[p_,q_,label_] := Graphics@AArrow[{p,q},label];

vec[p_] := Which[ Length[p] == 2, vec2[p],
                  Length[p] == 3, vec2[p] /. Graphics -> Graphics3D ];
vec[p_,q_] := Which[ Length[p] == 2, vec2[p,q],
                     Length[p] == 3, vec2[p,q] /. Graphics -> Graphics3D ];
vec[p_,q_,label_] := Which[  Length[p] == 2,vec2[p,q,label],
                             Length[q] == 3,vec2[p,q,label] /. Graphics -> Graphics3D];
sty[c__:Red] := (# /. Arrow[x__] -> {c, Arrow[x]})&
(*  sty[Red]@vec[{3,5}]*)

matrix[expr_] := expr /. List[p__]-> MatrixForm[List[p]]
cout[x__] := TeXForm[Row[{x}]];
pnt[x_] := Graphics3D@Point[x];

(* polynomial *)
(* helper functions *)
 vars[n_, m_] := Flatten@Transpose[Outer[Symbol@StringJoin[##] &, CharacterRange["A", "Z"][[;; m]], ToString /@ Range[n]]]
 polyvar[v_] :=  Flatten[{1,vars[v-1,1]}]; 

(* Give a list of coefficents and it will generate a polynomial with variables *)
poly[coef_] := Transpose[coef].polyvar[Length@coef];
poly@Thread[{{1,2,3}}];

T := Transpose;
Dim = Dimensions;
Ones[n_] := ConstantArray[1,n]
addCol[x_] := MapThread[Append, {#, x}] &
(*  addCol[ConstantArray[1,2]]@{{1,3},{3,4}} // matrix*)
addRow[x_] := Append[#,x]&

Convex Combination: Linear combination of vector with coefficients that add up to 1.
- Centroid: When the coefficients are equally distributed across vectors
The set of all convex combinations AKA if we varied the coefficients will be the Convex Hull of the component vectors.
- The center of the Convex Hull is the Centroid
We can build the Convex Hull by creating all combinations of triangles $\binom{|Vectors|}{3}$
- In our 3d plot, 4 vectors mean (4 choose 3) = 4 triangles


a = {1,3,3};
b = {5,2,2};
c = {4,2,1};
d = {3,3,3};
centroid[x__] = 

Row[{
    Show[
      vec[a,"a"],
      vec[b,"b"],
      sty[Dashed]@vec[(1-0.3)a+(0.3)b,"u1"],
      sty[Blue,Dashed]@vec[(1-0.5)a+(0.5)b,"centroid"],
      sty[Dashed]@vec[(1-0.7)a+(0.7)b,"u3"],
      sty[]@vec[a,b,"Convex comb"],
      ImageSize->Small,
        Axes->True],
        
    Show[
      vec[a,"a"],
      vec[b,"b"],
      vec[c,"c"],
      vec[d,"d"],
      Graphics3D[{Opacity[0.3],Polygon[{a,b,c,d}] }],
      Graphics3D@Text["Naive polygon method \n does not span the combination",{2,2,2}],
      ImageSize->Small,
        Axes->True],
    
    Show[
      vec[a,"a"],
      vec[b,"b"],
      vec[c,"c"],
      vec[d,"d"],
      sty[Blue,Dashed]@vec[(1/4)*a + (1/4)*b + (1/4)*c + (1/4)*d,"centroid"],
      Graphics3D[{Opacity[0.3],Polygon[{a,b,c}] }],
      Graphics3D[{Opacity[0.3],Polygon[{a,b,d}] }],
      Graphics3D[{Opacity[0.3],Polygon[{a,c,d}] }],
      Graphics3D[{Opacity[0.3],Polygon[{b,c,d}] }],
      ImageSize->Small,
        Axes->True]
    
    }]

0.1 Correlation

\[ Cos(\theta) = \frac{u}{|u|} \cdot \frac{v}{|v|} \in [-1,1]\] \[ Cos(\theta) \approx r = Correlation Coefficient \]

r $\approx$ Cos($\theta$)	meaning
0	90 degrees aka orthogonal
1	0 degrees aka same dir
-1	180 degrees aka opposite dir

\[r = \frac{\sum (u_{t} - \bar{u})(v_{t}-\bar{v})}{\sqrt{\sum(u_{t}-\bar{u})^2}\sqrt{\sum(v_{t}-\bar{v})^2}}\]

$u_{t} - \bar{u}$ centers the average $u$ to 0 and likewise for $v$
- This “centering the means to 0” is the only diff between Cos angle equation and Correlation Coefficient

Feature Vectors

Correlation coefficient is just the cos of the angle between 2 feature vectors.
- feature vectors means each vector represent a feature NOT a datapoint.
  - The dimension of a feature vector is the number of datapoints
- This is easy to visualize with only 2 data points but with n-data points the concept of angle in the n-th dimensional space is harder to visualize.

Notice the cos of the angle between 2 vectors are just the dot product of the normalized unit vectors.

Note that we also need to recenter (aka shift the x’s and y’s so the avg(x) = 0 and avg(y) = 0)

\[\text{corr coeff r} = Datapoints \rightarrow Feature\ Vector \rightarrow Recenter \rightarrow Normalize \rightarrow Get\ Angle\]

pt1 = {1,3};
pt2 = {3,4};
pt3 = {4,5};
x = {1,3,4};
y = {3,4,5};
recenterX = x - Mean[x];

cout["x-feature vector of datapoints ", x ,"is recentered as", recenterX , "to have avg weight as 0"]
Mean[recenterX];
recenterY = y - Mean[y];
cout["y-feature vector of datapoints ",y," is recentered as ", recenterY," to have avg weight as 0"]
cout[{pt1,pt2,pt3} // MatrixForm, "is now ",Thread[{recenterX,recenterY}], " but remember we are only interested in the feature vectors ", recenterX //MatrixForm, recenterY //MatrixForm ]

$$\text{x-feature vector of datapoints }\{1,3,4\}\text{is recentered as}\left\{-\frac{5}{3},\frac{1}{3},\frac{4}{3}\right\}\text{to have avg weight as 0}$$

$$\text{y-feature vector of datapoints }\{3,4,5\}\text{ is recentered as }\{-1,0,1\}\text{ to have avg weight as 0}$$

$$\left( \begin{array}{cc} 1 & 3 \\ 3 & 4 \\ 4 & 5 \\ \end{array} \right)\text{is now }\left( \begin{array}{cc} -\frac{5}{3} & -1 \\ \frac{1}{3} & 0 \\ \frac{4}{3} & 1 \\ \end{array} \right)\text{ but remember we are only interested in the feature vectors }\left( \begin{array}{c} -\frac{5}{3} \\ \frac{1}{3} \\ \frac{4}{3} \\ \end{array} \right)\left( \begin{array}{c} -1 \\ 0 \\ 1 \\ \end{array} \right)$$

angradian[u_,v_] = InverseFunction[Cos][(v.u)/(Norm[v]*Norm[u])];
anglebtw[x_,y_,l_:1,xbasis_:{1,0}] := Graphics@Circle[{0,0},l,{
                                            Min[angradian[x,xbasis],angradian[y,xbasis]], 
                                            Min[angradian[x,xbasis],angradian[y,xbasis]] + angradian[x,y]}]

Row[{
    Show[
      vec[pt1,"pt1"],
      vec[pt2,"pt2"],
      vec[pt3,"pt3"],
      
      anglebtw[pt1,pt2,2.1],
      ImageSize->Small,
        Axes->True],
    
    Show[Graphics@Text["visualized as recentered datapoints \nbut this isnt really \nthe graph to represent corr coeff r",{0,0.5}],
      vec[#]& /@ Thread[{recenterX,recenterY}] ,

      ImageSize->Small,
        Axes->True],
        
    Show[
    Graphics3D@Text["feature vectors",{0.2,0.5,0.8}],
      vec[recenterX,"x"] ,
      vec[recenterY,"y"] ,
      ImageSize->Small,
        Axes->True]
    
          }]

The second graph is incorrect in visualizing the correlation coefficient because they represent datapoints.
Correlation coefficient r is represented by the angle between 2 feature vectors correctly shown in the 3rd 3d graph
We just happened to pick a good dataset that can be visualized in the last 3d graph. Our dataset has 3 datapoints with 2 features (x,y).
- The extracted x features from the 3 datapoints is represented as a 3d vector
- The same can be said of the extracted y features

What if:

Notice if we had 4 datapoints, it would show a 4 dimensional graph(cant visualize).
If we had 3 features (x,y,z), it would show 3 vectors.
- Correlation coefficient r works only on 2 features; we can have multiple correlation coefficient $\binom{|features|}{2}$ more precisely.

0.2 Equation of planes

equation form
Parametric form
point normal vector form
3 points determine plane form

0.2.1 Equation form

beta := {1,2,3}
x := {x1,x2,x3}
subst = Solve[x.beta == 4,x3];
sol = x3 /. subst;
cout["Equation form " , x.beta == 4]
cout["solution is rule ", subst, " applied to ", x, " is sol ", sol ]

$$\text{Equation form }\text{x1}+2 \text{x2}+3 \text{x3}=4$$

$$\text{solution is rule }\left\{\left\{\text{x3}\to \frac{1}{3} (-\text{x1}-2 \text{x2}+4)\right\}\right\}\text{ applied to }\{\text{x1},\text{x2},\text{x3}\}\text{ is sol }\left\{\frac{1}{3} (-\text{x1}-2 \text{x2}+4)\right\}$$

Row[{

   
    Show[
        vec[pp,pp+vk1,"P+ta"],
        vec[pp,pp+vk2,"P+eb"],
        vec[pp,pp,"P"],
        ImageSize->Small,
        Axes->True
        ]
    }
    ]

0.2.2 Point normal form


point := {1,0,1}
normal := {1,2,3}
x := {x1,x2,x3}
subst := Solve[x.(point - normal) == 0]
sol = x3 /. subst;
Show[
    vec[point,point,"Point P"],
    vec[normal,"normal vect"],
    Plot3D[sol, {x1,-2,2},{x2,-2,2},PlotStyle->Opacity[0.2]],
    ImageSize->Small,
        Axes->True
  
]

cout["All we needed to define the plane was a point on the plane ", matrix@point , " and a normal vector to the plane ", matrix@normal ]
cout["the solution to ", matrix[Inactivate[x.(point - normal) == 0,Dot]] , " is ",subst , " so we plot ", matrix@(x /. subst)]

$$\text{All we needed to define the plane was a point on the plane }\left( \begin{array}{c} 1 \\ 0 \\ 1 \\ \end{array} \right)\text{ and a normal vector to the plane }\left( \begin{array}{c} 1 \\ 2 \\ 3 \\ \end{array} \right)$$

$$\text{the solution to }\left( \begin{array}{c} \text{x1} \\ \text{x2} \\ \text{x3} \\ \end{array} \right).\left( \begin{array}{c} 0 \\ -2 \\ -2 \\ \end{array} \right)=0\text{ is }\{\{\text{x3}\to -\text{x2}\}\}\text{ so we plot }\left( \begin{array}{ccc} \text{x1} & \text{x2} & -\text{x2} \\ \end{array} \right)$$

0.2.3 Parametric Form

pp = {1,0,1};
vk1 := {2,-1,0}
vk2 := {0,3,-2}
sol = poly[{pp,vk1,vk2}];
cout["P is ", matrix @ pp, ", vk1 is ", matrix@ vk1, ", vk2 is ", matrix @ vk2]
cout[matrix@Inactivate[1*pp + A1*vk1 + A2*vk2,Plus | Times] == matrix@poly[{pp,vk1,vk2}]]
cout["Varying scalar coefficients A1, A2 creates a spanning plane with vk1,vk2 vector as basis"]
cout["P vector is like the y-intercept"]

$$\text{P is }\left( \begin{array}{c} 1 \\ 0 \\ 1 \\ \end{array} \right)\text{, vk1 is }\left( \begin{array}{c} 2 \\ -1 \\ 0 \\ \end{array} \right)\text{, vk2 is }\left( \begin{array}{c} 0 \\ 3 \\ -2 \\ \end{array} \right)$$

$$1\times \left( \begin{array}{c} 1 \\ 0 \\ 1 \\ \end{array} \right)+\text{A1}\times \left( \begin{array}{c} 2 \\ -1 \\ 0 \\ \end{array} \right)+\text{A2}\times \left( \begin{array}{c} 0 \\ 3 \\ -2 \\ \end{array} \right)=\left( \begin{array}{c} 2 \text{A1}+1 \\ 3 \text{A2}-\text{A1} \\ 1-2 \text{A2} \\ \end{array} \right)$$

$$\text{The scatter plot shows us varying scalar coefficients A1, A2 creates a spanning plane}$$

    Show[
        ListPointPlot3D[Table[sol,{A1,0,2,0.2},{A2,0,2,0.2}]],
        sty[Blue,Thickness[0.01]]@vec[pp,pp+vk1,"P+vk1"],
        sty[Blue,Thickness[0.01]]@vec[pp,pp+vk2,"P+vk2"],
        vec[pp,pp,"P"],
        ImageSize->Small,
        Axes->True
        ]