1) e^x is a function whose derivative is equal to its value.
2) e^ix is a function whose derivative is equal to its value rotated by 90 degrees (ie^ix).
3) As x goes from 0 to pi, the trajectory of e^ix always has a velocity vector perpendicular to its current position. For example, when x = 0, the current position is 1 and the velocity vector is i.
4) So the trajectory a circle arc of length pi, which ends at -1.
In a linear algebra course I helped teach last semester, I had the students go through the exercise of writing down the matrix for "multiplication by i," where the complex numbers are thought of as a two-dimensional vector space with {1,i} as a basis. Then I asked them to recognize the matrix (it's a 90-degree rotation of the plane), and then I asked them for an interpretation of that matrix squared (a 180-degree rotation). One thing to draw from this: for the real line, negation doesn't reflect the line, but instead it's a rotation through another dimension of numbers!
When we went on to differential equations, I tried to convince them that e^(ix) is meaningful using your #2: it is a path around the unit circle since it's the solution to z'=iz, and we already agreed multiplication by i is a 90 degree rotation, and circles are what you get when velocity is orthogonal to vector position. It's a bit tougher to convince someone that x is arclength, though, especially when they aren't too comfortable with complex numbers yet.
the standard IEEE floating point uses a 'sign and magnitude' system. Negative numbers are basically made by flipping a single bit. I've been playing around with a number system that expresses real numbers as two's complement (like how integers work) - xor all your bits and then add one. In this system, 0 is -0 and the number that is weird for signed integers: 0b100000...0000 is infinity.
Now, there's an exponent and a fraction, too. I've been playing around with how to do these numbers with logic gates (and verilog!) and you can either two's complement the whole thing and work with absolute values, or you can keep the fraction part as a two's complement...
So I just redid multiplication using two's complemented fractions! And addition/subtraction too, which for floating points in general is significantly harder than multiplication. The nice thing about two's complemented floating points is you don't need separate algorithms for addition and subtraction; you can just do everything with one algorithm.
Well, I'm the guy in the black shirt who did the demo.
If you liked that lecture, I'm already starting on some verilog implementations.
This is an example multiplication 8 bit * 8 bit -> 16 bit unpacked (20 bits). It differs from standard floating point in that the fractions are stored as two's complement. It takes a little bit of wrapping your head around, but the hidden bit for negative numbers is actually -2 ! Moment of zen.
Way cool -- I realized that when I checked out your profile just after posting the comment -- what timing -- I discovered the video a few days ago when looking for precise/compact interval representations. Interesting work indeed.
Has there been much traction for getting major chip manufacturers to implement this? I know they're all looking for the next big thing and Intel is working on specialized neuromorphic chips. A general "drop-in" replacement for floating point seems like an opportunity for a general-purpose win from low-hanging fruit:
well seeing as how John invented these numbers literally two months ago, I haven't seen any traction yet! But I am persuing fundraising opportunities. In the demo I showed how you can effectively reduce the bitwidth to 8 bits and still train in a very trivial machine learning exercise. I'm currently enrolled in the udacity machine learning class and implementing everything in parallel in julia so that I can try more complicated architectures using posits.
I do have a hardware architecture in mind for how to very effectively and efficiently execute machine learning calculations using posits.
I’m interested in representing angles / points on the circle; 3d unit vectors / points on the sphere; unit quaternions / points on the 3-sphere using 1, 2, or 3 relatively low-resolution posits, under stereographic projection.
How efficient do you think regular C or GPU code (on existing hardware) can be made for compressing a 32-bit float to e.g. a 16-bit posit, and for expanding the posit back into a 32-bit float?
I don't think it can be made that efficient in software. Is there a particular reason why you need 16 bits? A 32-bit float is going to be better than a 16-bit posit almost always (posits are better their equivalently sized float, but they're not that good) and once it's in posit representation do you have a way of doing mathematical operations on them?
This is just for data compression, not for computation directly. 32 bits is often overkill for transmission/storage of rotations, unit vectors, geographical locations, unit quaternions, and the like. Depending on the use case 8 bits might be enough, or 12, or 16.
To actually do computation I would convert the posits back into 32-bit floats (or e.g. in the Javascript case, 64-bit floats), and then take the inverse stereographic projection.
[Stereographic projection is extremely cheap; for each data point only requires one division and some additions and multiplications.]
Posits look too good to be true!
Is there a reason why regime bits do not include the sign bit?
Then both "0 0001" and "1 1110" could be interpreted as 4 regime bits. Even better we could include the last flipped bit as well and we would have 5 bits.
Edit: Well. I see it would result in losing the values 0 and 1.
Another question:
Since it is fixed length of 4 bits (for N=32) why don't we just extract the 4 bit value, then we could represent 2^4 regimes this time without losing 0 and 1.
I wouldn't screw around too much with the sign bit. The way it's laid out is really kind of cool... Negation is simple two's complement.
In my software posit library (which is intentionally strictly binary and not backended by IEEE floats), (https://github.com/interplanetary-robot/SigmoidNumbers) I did everything by first inverting negative numbers and doing decode in the positive domain.
As I design the hardware, it's actually better to NOT do a two's complement inversion to do the decode, and keep the fraction as two's complement!
Also the 4 bit posit was just a simplification to help you understand the structure from a constructive point of view. posits can be of arbitrary length; they have a property I call isomorphic - so appending zeros exactly preserves the value of a short posit when increased in length; conversely, rounding a long posit to a shorter one reports the "nearest representable value".
You need to be a lot more explicit about what you’re asking.
Look at the slides, searching down for “At nbits = 5, fraction bits appear”. Notice that every possible bit pattern is used and meaningful.
If you are unsure why you were downvoted, I strongly suspect it's because politely giving and receiving due credit are important, knowing that one's interlocutor was involved in specific research is very useful information (to know what questions to ask), and it's probably kind of rude to call someone to account in this manner. Hope this helps :)
You are mixing up a couple of things. This "dimension" you're talking about is probably the dimension of a vector space. The real numbers, as a vector space over the field of real numbers (recall that vector spaces are defined over fields of numbers), have dimension 1. However, you can consider the vector space of real numbers over the field of rational numbers. What is the dimension of this vector space? Well, the dimension of a (finite-dimensional) vector space is given by the number of elements in a basis for it. So how do we go about finding a basis for R over Q? A basis for R over R consists solely of the number "1", since given a real number "x" there exists an element from the field (in this case R), namely x itself, such that multiplying it by 1 will give you the number x. Yes, this sounds obvious, but that's how you prove that R over R has dimension 1. So, going back to R over Q, we see that "1" cannot be a basis, since if we pick, say, pi, there is no rational that we multiply 1 by to give pi. Moreover, there cannot be any finite number of real numbers that would make up a basis for R over Q, since then R would be countable (look this up if you don't know what it is). So the dimensionality of a vector space depends on which field you're considering. By the way, the vector space of complex numbers over the reals has dimension 2 (a basis is {1, i}), but over the complex numbers it has dimension 1. So, there's an "extra" dimension only if you consider it over the reals.
C (complex numbers) is a field, just like R (real numbers). It's an algebraically closed field, unlike R, so in some sense it's actually the most natural set to call "numbers".
Well I was answering the question about why is C considered numbers - it's a field, hence elements of C behave exactly like numbers. Moreover, it's an 'algebraic closure' of R, hence a more natural choice for "all numbers", and it's a maximal one at that (i.e. quaternions and such are no longer fields and don't really behave like numbers, while C still does).
Edit: FWIW, I consider the terminology of 'real' vs 'imaginary' completely stupid and misleading. This terminology didn't really make it to other languages, e.g. in Russian it's 'material' vs 'complex' numbers, but they don't use the term 'imaginary'.
This isn't a proof, and while that may be obvious to you, it's not obvious to everyone who sees it.
There exists no logical path to the statement from the previous definition of e^x because, up to this point, e^x is a function on real numbers. e^ix is nonsense until you define e^x in complex coordinates, at which point the proof needs to rely on properties of that definition.
what you have posted misses the nature of the insight.
If two analytic functions on the complex numbers agree on uncountably infinitely many points, then they agree everywhere they are defined. This means that there is a unique analytic function that maps (ix)[x \in R] to the circle and (x)[x \in R] to the natural exponential, and they are the same complex function. Without the knowledge that analytic continuations are unique, the statement is entirely (pun intended) meaningless.
Sure, step 1 relies on the existence of e^z which is not obvious. Step 2 relies on the chain rule over C, step 4 relies on having a unique solution to a differential equation, etc. My comment can be expanded to a proof, but it was more aimed to explain why Euler's identity is natural.
step 1 isn't "not obvious"; it's nonsense. What, exactly, are you trying to prove exists? e^z? e^z is undefined at this point. If you're trying to prove that e^i\pi=-1, you need to start with a definition of e^z.
Minor wording: e^x is a function whose derivative is equal to itself, or the value of whose derivative is equal to its own value. This might be considered the kind of excessive precision that obscures rather than clarifies (as would be, for example, writing x ↦ e^x in place of just e^x), but lots and lots of confusion can result (especially in a function-analytic setting) from failing to distinguish between a function and its values.
I can't tell if you are being sarcastic, but no, parent's comment is not the contents of the video. The video provides a geometric construction, and makes no use of calculus.
For me what made it clicked is realizing that complex numbers are 2D matrices: z = x + i y = [[x -y][y x]].
So really we should be writing z = x * [[1 0][0 1]] + y * [[0 -1][1 0]], but since it's tedious we just call 1 == [[1 0] [0 1]] the 2x2 identity matrix and i == [[0 -1][1 0]], and check that i^2 = -1.
Then no more magical i number, the complex product can be derived from the matrix product, the exponential becomes the 2x2 matrix exponential, and so on.
Using a matrix as an exponent isn't any more comprehensible than an imaginary number to me. If it works for you, that's great, but it's not much help to me.
Think of exponentiation of some number 'a' as in-between its integer powers: 'a^1.5' is kind-of half-way between 'a' and 'a^2'.
If you plot all the integer powers of 'a', they all belong to a curve and the exponential simply fills-in the gaps for non-integer exponents.
Now, there are many possible ways to fill the gaps but the exponential does it so that a^m * a^n = a^{m+n} holds even for non-integer numbers m and n.
Similarly, if you take integer powers of a complex number, they all lie on some curve and the exponential fills-in the gaps, again turning sums into products. The same works with matrices, and so on.
The complex plane can be thought of as being made up of two-dimensional geometric transformations consisting of rotation and scaling (Tristan Needham calls such transformations “amplitwists”), with 1 as the identity transformation, and i as a quarter turn anticlockwise, and i^2 = –1 as a half turn. To compose two such transformations, you multiply the scales and add the angle measures of the rotations. Because such operations are linear, you can also break them into a part parallel to 1 and a part perpendicular to 1 (some multiple of i), and multiply two such transformations component-wise, using the distributive law (a + bi)(c + di) = (ac – bd) + (ad + bc)i.
exp z is a complex function which maps (in an angle-preserving way, i.e. conformally) from an infinite two-ended cylinder to a whole plane minus one point (the “origin”). The exp function maps negative infinity on the cylinder to the origin on the plane, and it maps the zero point on the cylinder to a given “unit” point in the plane, and the “zero” circular slice through that point on the cylinder to the “unit circle” on the plane, containing the unit point and concentric with the origin. The coordinate system on the cylinder has 2πi measuring one loop around a circular slice, and 1 pointed along the cylinder axis. The coordinate system in the plane is the customary square grid. Addition of coordinates in the geometry of the cylinder (if you like, rotating and/or sliding the cylinder) corresponds to multiplication of complex numbers (composition of amplitwists) in the plane. That is, exp(w + z) = (exp w)(exp z).
In particular, exp iy for some real number y maps points at distance y along the zero circle in the cylinder to points on the unit circle in the complex plane at a proportional distance around the circle; that is, to rotation operators which correspond to the given angle measure in radians.
So πi is halfway around the zero slice in the cylinder, and exp maps it to the operator in the complex plane corresponding to a half-turn rotation, i.e. exp πi = –1.
The log function is the inverse map, from the plane to the cylinder; it is a multi-valued function because we can make multiple “straight” helical connections between arbitrary points on the cylinder, which wrap around different numbers of times.
Once we have this general concept for how we want the exp map to work, we can work out the details to find that the unique such function is the solution to a particular differential equation f'(z) = f(z), or alternately the Taylor series we are all familiar with, exp z = 1 + z + z^2/2 + z^3/6 + ...
2) e^ix is a function whose derivative is equal to its value rotated by 90 degrees (ie^ix).
3) As x goes from 0 to pi, the trajectory of e^ix always has a velocity vector perpendicular to its current position. For example, when x = 0, the current position is 1 and the velocity vector is i.
4) So the trajectory a circle arc of length pi, which ends at -1.