RNN - Week 1
The first week covers different types of RNNs and the basic concept of feeding forward the outcome of previous steps in a sequence to the next.
Different types of RNNs
- Many-to-many, multiple inputs and outputs. This can be split into two types, where input and output is of same size or not.
- Many-to-one, multiple inputs, one output.
- One-to-one, Standard neural network.
- One-to-many, one input, multiple outputs.
- Attention based, to be explored in week 4…
Usecases well suited for RNNs
- Input and output lengths might vary
- Sharing features learned for certain positions across the full input
- With a large vocabulary, the amount of parameters becomes very large for traditional networks
A side note is that the two last points sounds like something that 1D convolution also might reduce, but hopefully it will become clearer as the course goes on how they would or wouldn’t work in the use cases covered by RNNs.
More …