This circle has the centre at the origin and a radius of 1 unit. The point P can move around the circumference of the circle. At point P the \(x\)-coordinate is \(\cos{\theta}\) and the ...

This circle has the centre at the origin and a radius of 1 unit. The point P can move around the circumference of the circle. At point P the \(x\)-coordinate is \(\cos{\theta}\) and the ...

Hi, thanks for the great work! I was reading the SinusoidalPosEmb implementation, and I noticed that the sinusoidal encoding is constructed by concatenating sin and cos along the last dimension: emb = ...