In a medium saucepan, combine the pumpkin puree, water, and brown sugar and bring to a simmer for 3-5 minutes over medium heat.
Remove from the heat and over a bowl strain the mixture through a strainer, ensuring to remove the pumpkin pulp. Using a rubber spatula, press out as much of the liquid from the pulp of the pumpkin.
In the same saucepan, add the strained pumpkin liquid and the can of sweetened condensed milk. Mix until thoroughly combined, this will take some time to combined fully.
Add in the ground cinnamon, pumpkin pie spice, and vanilla bean paste (or vanilla extract). Stir to completely combine.
Remove from the heat and let cool for about 10 minutes.
Store in mason jars, squeeze bottles, or an airtight container.
Enjoy!