Robots are increasingly deployed into safety-critical applications. However, safe navigation remains a challenge due to uncertain vehicle dynamics and imperfect controllers. To handle safety, we often inflate obstacles or craft safety tubes around trajectories. Experts hand-tune static safety margins for particular missions, however this is valid only under low dynamics variation. Conversely, one can use worst-case margins by assuming high dynamics range, but overly conservative approaches can lead to suboptimal solutions. We propose a middle ground: margins that adapt on-the-fly with online measurements. To enable real-time adaptation, our approach precomputes a library of safety tubes at different levels of dynamics uncertainty. Online, our system queries appropriate safety margins based on its estimated dynamics uncertainty for safe real-time planning. We demonstrate with real flight tests that we can safely capture unknown varying dynamics without overly sacrificing performance. Experimental results show that our method computes more optimal plans while remaining safe compared to baseline static margin methods.