What is the difference between a private key and a seed?
When people talk about the "secret key", do they mean the "private key? Are they the same?
What are public keys? How are they related to addresses?
The Private/Public key pair (aka asymetric cryptography) refers to a common cryptographic mechanism. Private and public keys are bounds together and also bound to a particular well known cryptographic algorithm like 'RSA', 'SHA', ... Private and public key aren't just random string, in fact they have the following very interesting property :
If you receive a piece of data + a signature for this piece of data, then the public key allows you to check that the signature was generated with the corresponding private key and for this exact piece of data. (thanks to the well known cryptographic algorithm)
In other words the public key allows you to check that the emitter of the data is the owner of the private key, and that the data was not modified.
How is it use in IOTA ?
In IOTA (like in many other crypto-currencies), an address is a public key, you can disclose it, it is not a secret.
When you spend some IOTAs that are in a particular address, you need to demonstrate to the rest of the world that you are the owner of the address (i.e. the owner of the private key for that public key). To do that you sign the transaction with the private key associated with that address. The rest of the world know the public key (it's the address), and your signature is in the transaction. The rest of the world can easily check that you are the owner of the address and that the content of the transaction wasn't modified by some third party.
In IOTA: the private/public key pair is generated deterministically from your seed + some index (+ a security level, but it is not specially relevant here). It means that :
So, your seed is kind of "randomness" that allows generation of an almost infinite amount of private/public key-pair.
To answer a side question: why can't we reuse the same address for multiple withdraw ?
To understand the reason we have to look deeper into the "well known cryptographic algorithm" used on the tangle. Iota use a (variation of) the Lamport algorithm.
The major advantage of this algorithm is that it is expected to be quantum resistant (while all other currently well known cryptographic algorithms will be useless when quantum computers will be a reality)
Unfortunately, this "quantum resistance" have a cost: the Lamport algorithm is forcing the disclosure of 50% of the private key in the signature. (for a deeper understanding of the algorithm itself : https://en.wikipedia.org/wiki/Lamport_signature).
So this algorithm is a "one-time-signature", because if you use it twice: you are going to reveal more than 50% of your private. An attacker having 2 or more signatures generated from the same private key will be able to generate a transaction with a a signature that could have been produced by your private key (the time to produce such a transaction depends on the disclosed the portion of the key). This is considered as a risk.
The Seed is your only local stored authentification and is never revealed in a transaction.
Private Key's are generated with your seed in combination with a key index and are needed to sign the transaction. Random 50% of the private key are leaked when signing a transaction. That's why u should never reuse an Adress. In the most cases Private key is used as a synonym to Secret key.
The Public key is a synonym for the Adress, which is generated from the Private key.
Further reading: Seeds, Private Keys and Adresses