$$C$$ = Center of the elliptic cone in world space
$$H$$ = Point at the tip of the elliptic cone in world space
$$\mathbf{a}$$ = First elliptic cone axis vector (along local x axis)
$$\mathbf{b}$$ = Second elliptic cone axis vector (along local y axis)
$$\mathbf{c}$$ = Elliptic cone height vector (along local z axis)

$$L_{0}$$ = Point on the line in world space
$$\mathbf{v}$$ = Vector that defines the line direction in world space

$$P$$ = Point on the surface of a unit radius and height cone with the tip at the origin
Transform matrix from unit radius and height cone with the tip at the origin to non uniformly scaled cone A simpler way to calculate the intersection of a line and a cone arbitrarily scaled in space is to build the matrix that transforms from the scaled cone to a unit radius and height cone with the tip at the origin. Apply the transformation to the line, calculate the intersection with a unit radius and height cone, then apply the results to the original line and obtain the intersection points if any are found.

The transform matrix is the result of the composition of the matrices that describe translation, rotation and scale of the scaled cone. Its inverse maps the non uniform scaled space to the uniform one. $\mathbf{T} = \begin{bmatrix} 1 & 0 & 0 & H_{x} \\ 0 & 1 & 0 & H_{y} \\ 0 & 0 & 1 & H_{z} \\ 0 & 0 & 0 & 1 \end{bmatrix} \qquad \mathbf{R} = \begin{bmatrix} \hat{a}_{x} & \hat{b}_{x} & \hat{c}_{x} & 0 \\ \hat{a}_{y} & \hat{b}_{y} & \hat{c}_{y} & 0 \\ \hat{a}_{z} & \hat{b}_{z} & \hat{c}_{z} & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \qquad \mathbf{S} = \begin{bmatrix} \left\lVert \mathbf{a} \right\rVert & 0 & 0 & 0 \\ 0 & \left\lVert \mathbf{b} \right\rVert & 0 & 0 \\ 0 & 0 & \left\lVert \mathbf{c} \right\rVert & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$ $\mathbf{M} = \mathbf{TRS}$ Point on unit radius and height cone with the tip at the origin A generic point on the cone satisfies the equation: $\frac{\left\lVert P - Q \right\rVert}{\left\lVert Q - H' \right\rVert} = \frac{r}{\left\lVert C - H' \right\rVert}$ Where $$Q$$ is the projection of $$P$$ on the cone axis.

Both sides of the equation can be raised by the power of 2, to simplify the calculations. $\frac{\left\lVert P - Q \right\rVert^{2}}{\left\lVert Q - H' \right\rVert^{2}} = \frac{r^2}{\left\lVert C - H' \right\rVert^{2}}$ The rescaled cone tip $$H'$$ is in the origin and its base center $$C'$$ is one unit farther along the negative Z axis. The radius is equal to 1: $H' = H \mathbf{M}^{-1} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \qquad C' = C \mathbf{M}^{-1} = \begin{bmatrix} 0 \\ 0 \\ -1 \end{bmatrix} \qquad r = 1$ The height vector $$\mathbf{h}$$ is defined by the difference of $$H'$$ and $$C'$$. $\qquad \mathbf{h} = C' - H' = \begin{bmatrix} 0 \\ 0 \\ -1 \end{bmatrix} \qquad \left\lVert \mathbf{h} \right\rVert = 1$ Substitute in the original equation. $\frac{\left\lVert P - Q \right\rVert^{2}}{\left\lVert Q - H' \right\rVert^{2}} = 1$ $\left\lVert P - Q \right\rVert^{2} = \left\lVert Q - H' \right\rVert^{2}$ The segment $$\overline{H'Q}$$ can be expressed in terms of $$P$$ as: $\left\lVert P - Q \right\rVert^{2} = \left\lVert P - H' \right\rVert^{2} - \left\lVert Q - H' \right\rVert^{2}$ $\left\lVert Q - H' \right\rVert^{2} = \left[ \left( P - H' \right) \cdot \mathbf{h} \right]^{2}$ Substitute in the original equation: $\left\lVert P - H' \right\rVert^{2} - \left[ \left( P - H' \right) \cdot \mathbf{h} \right]^{2} = \left[ \left( P - H' \right) \cdot \mathbf{h} \right]^{2}$ Group the equal terms. $\left\lVert P - H' \right\rVert^{2} = 2 \left[ \left( P - H' \right) \cdot \mathbf{h} \right]^{2}$ Parametric line A generic line defined by a point $$L_{0}$$ and a direction vector $$\mathbf{v}$$. It describes all the points along the line as a function of the parameter $$t$$. $L_{\left( t \right)} = L_{0} + t \mathbf{v}$ To perform the intersection with the unit radius and height cone with the tip at the origin, the point $$L_{0}$$ and the vector $$\mathbf{v}$$ are transformed by the matrix $$\mathbf{M}^{-1}$$. $L'_{\left( t \right)} = L_{0} \mathbf{M}^{-1} + t \mathbf{v} \mathbf{M}^{-1}$ $L'_{0} = L_{0} \mathbf{M}^{-1} \qquad \mathbf{v}' = \mathbf{v} \mathbf{M}^{-1}$ $L'_{\left( t \right)} = L'_{0} + t \mathbf{v}'$ Intersections calculation To calculate the intersection between cone and line, substitute the generic point $$P$$ on the cone with a generic point on the line $$L'_{(t)}$$. $\left\lVert L'_{0} + t \mathbf{v}' - C' \right\rVert^{2} - \left[ \left( L'_{0} + t \mathbf{v}' - C' \right) \cdot \mathbf{h} \right]^{2} = 1$ Define a vector $$\mathbf{w}$$ equal to the difference of $$L'_{0}$$ and $$C'$$. $\mathbf{w} = L'_{0} - C'$ and substitute back. $\left\lVert t \mathbf{v}' + \mathbf{w} \right\rVert^{2} - \left[ \left( t \mathbf{v}' + \mathbf{w} \right) \cdot \mathbf{h} \right]^{2} = 1$ The length squared of a vector is equal to the dot product of the vector by itself. $\left( t \mathbf{v}' + \mathbf{w} \right) \cdot \left( t \mathbf{v}' + \mathbf{w} \right) - \left[ \left( \mathbf{v}' \cdot \mathbf{h} \right) t + \left( \mathbf{w} \cdot \mathbf{h} \right) \right]^{2} = 1$ Expand the dot product and the power. $\left( \mathbf{v}' \cdot \mathbf{v}' \right) t^{2} + 2 \left( \mathbf{v}' \cdot \mathbf{w} \right) t + \left( \mathbf{w} \cdot \mathbf{w} \right) - \left( \mathbf{v}' \cdot \mathbf{h} \right)^{2} t^{2} - 2 \left( \mathbf{v}' \cdot \mathbf{h} \right) \left( \mathbf{w} \cdot \mathbf{h} \right) t - \left( \mathbf{w} \cdot \mathbf{h} \right)^{2} = 1$ Group by the coefficients of $$t$$. $\left[ \left( \mathbf{v}' \cdot \mathbf{v}' \right) - \left( \mathbf{v}' \cdot \mathbf{h} \right)^{2} \right] t^{2} + 2 \left[ \left( \mathbf{v}' \cdot \mathbf{w} \right) - \left( \mathbf{v}' \cdot \mathbf{h} \right) \left( \mathbf{w} \cdot \mathbf{h} \right) \right] t + \left( \mathbf{w} \cdot \mathbf{w} \right) - \left( \mathbf{w} \cdot \mathbf{h} \right)^{2} - 1 = 0$ Substitute the coefficients of $$t$$ to obtain a simpler quadratic equation. $\begin{cases} a = \left( \mathbf{v}' \cdot \mathbf{v}' \right) - \left( \mathbf{v}' \cdot \mathbf{h} \right)^{2} \\ b = 2 \left[ \left( \mathbf{v}' \cdot \mathbf{w} \right) - \left( \mathbf{v}' \cdot \mathbf{h} \right) \left( \mathbf{w} \cdot \mathbf{h} \right) \right] \\ c = \left( \mathbf{w} \cdot \mathbf{w} \right) - \left( \mathbf{w} \cdot \mathbf{h} \right)^{2} - 1 \end{cases}$ Solve for $$t$$. $a t^{2} + b t + c = 0$ Intersection points
• if $$\left( b^{2} - 4 a c \right) > 0$$
There are two intersections at: $L_{int_{1,2}} = L_{0} + t_{1,2} \mathbf{v} \qquad t_{1,2} = \frac{ - b \pm \sqrt{ b^{2} - 4 a c } }{ 2 a }$

• if $$\left( b^{2} - 4 a c \right) = 0$$
The line $$L_{ \left( t \right) }$$ is tangent to the cone. $\left\lVert \mathbf{h} \right\rVert = cos \left( \alpha \right) \sqrt{\left\lVert \mathbf{h} \right\rVert^{2} + r^{2} } \qquad cos \left( \alpha \right) = \frac{\left\lVert \mathbf{h} \right\rVert}{\sqrt{\left\lVert \mathbf{h} \right\rVert^{2} + r^{2} }}$
• if $$\left| \mathbf{\hat{v}} \cdot \mathbf{\hat{h}} \right| \neq cos \left( \alpha \right)$$
There is one intersection at: $L_{int} = L_{0} + t \mathbf{v} \qquad t = - \frac{ b }{ 2 a }$

• if $$\left| \mathbf{\hat{v}} \cdot \mathbf{\hat{h}} \right| = cos \left( \alpha \right)$$
The line $$L_{ \left( t \right) }$$ lies on the cone surface. There are infinite intersections.

• if $$\left( b^{2} - 4 a c \right) < 0$$
There are no intersections.

If any intersections are found and the cone is not infinite, the projection of the intersections on the cone axis must be between the two extremities $$H$$ and $$C$$.

• if $$0 \leq \left( L_{int} - C \right) \cdot \mathbf{h} \leq \left\lVert \mathbf{h} \right\rVert$$
The intersection is on the cone surface.

• if $$\left( L_{int} - C \right) \cdot \mathbf{h} > \left\lVert \mathbf{h} \right\rVert$$
The intersection is below the base of the cone. Test the intersection with the cone base cap.

• if $$\left( L_{int} - C \right) \cdot \mathbf{h} < 0$$
The intersection is above the top of the cone.