The name of the papyrus plant travelled from Greek into Latin and then through French into English. Middle English papir is one stop on that journey. Paper keeps the plant’s name even when its fibres come from something entirely different.
These are selected stages, not every spelling or intermediary. The story identifies omitted links; older forms did not all disappear at once.