In the context of trying to fix something, it not working out and then having to go back over it, yeah, I'd recommend replacing them (quote from page 2 below). "...The separation of the gasket is not normal. It might also be acceptable? I have this as well on the gasket between my upper and lower plenum due to fucking with all that far too much but in my case and setup, it makes no difference. No vacuum leak. In your case, I'd say you'll be fine but honestly, I think you've been through enough bullshit to then have to re-do this once AGAIN, right? I'd get new gaskets. I replaced the bolts on the transmission pan when I installed a shift kit only because the FSM gave instruction to do so whereas most probably do not, so there you go."
Personally, I wouldn't replace air to air gaskets even if they failed the fix the first time around but if I did, I'd give those aftermarket gaskets a whirl. I've pulled my plenums many times (oem gasket still present) and with the cork gasket on the kinetix plenum, that's not a good idea unless one is careful and paying close attention to what one is doing. If an air to air type gasket wouldn't seal for me and it was clearly in tact, then there's a mating surface situation or all surfaces weren't clean or something along those lines. I'd permatex the shit out of it.
When I tucked my engine bay I removed the excess gasket material wherever I found it. Somehow, I managed to pull out a bit of the material between the block and water pump cover

. No big deal, but I thought I could make it right without pulling the cover. I cleaned that area extremely well with brake cleaner then acetone, focusing on the small gap left behind by the missing gasket maker then put some gasket maker on the tip of my finger, smashed it in there, wiped away the excess and no more leak. Took 10 minutes.
Foreign OEM has massive overhead cost and theft-like pricing. I say "like" because one doesn't have to buy their parts. Those aftermarket gaskets are probably just fine imo. Get a better name from rockauto or somewhere else online and you're still way ahead of oem.