warping with ffmpeg - and some filter_complex examples

Since OCVWarp uses OpenCV remap() and there does exist a remap filter in ffmpeg also, revisited ffmpeg remap in order to make use of the accelerated NVidia codecs, which ffmpeg supports. Asked ChatGPT to generate python code to create the required map files, debugged the code manually, and put the code at

https://github.com/hn-88/OCVWarp/tree/master/python

Currently it's just for the spherical mirror warping of fulldome content, but could potentially be extended to all of OCVWarp's functionality.

This ffmpeg command using the accelerated hevc_nvenc codec runs at 13 fps on a desktop running NVidia RTX 1060 as against 2.5 fps using OCVWarp saving to avc1 codec out.

ffmpeg -i input.mp4 -i map_x_directp2.pgm -i map_y_directp2.pgm -i weight_alpha_mask.png -filter_complex "
  [0:v]scale=3840:2160[scaled];
  [scaled][1:v][2:v]remap[remapped];
  [3:v]format=gray,scale=3840:2160,colorchannelmixer=rr=1:gg=1:bb=1[mask_rgb];
  [remapped][mask_rgb]blend=all_mode=multiply[out]
" -map "[out]" -map 0:a -c:v hevc_nvenc -preset p5 \

-cq 23 -rc vbr -maxrate 15M -bufsize 26M \
-c:a aac -b:a 128k output.mp4

Can potentially even create a UI using Tkinter in python, which would be cross-platform.

Some notes about the ffmpeg commands copy-pasted from the readme there - 

-ss for the start time should come before the input, and 
-t for the duration should come just before the filter_complex - 

For starting at two minutes and stopping after 10 seconds duration,

ffmpeg -ss 00:02:00 -i input.mp4 \

       -i map_x_directp2.pgm \

       -i map_y_directp2.pgm \

        -i weight_alpha_mask.png \

       -t 10 \

-filter_complex "

 [0:v]scale=3840:2160[scaled];

  [scaled][1:v][2:v]remap[remapped];

  [3:v]format=gray,scale=3840:2160,colorchannelmixer=rr=1:gg=1:bb=1[mask_rgb];

  [remapped][mask_rgb]blend=all_mode=multiply[out]

" -map "[out]" -map 0:a -c:v hevc_nvenc -c:a copy output.mp4

And a problematic video was fixed using an initial crop to 4096x4096 (although avidemux could show it as 4096x4096)

ffmpeg  -i "short_nometadata.mp4"        -i map_x_directp2.pgm        -i map_y_directp2.pgm         -i weight_alpha_mask.png      -filter_complex "[0:v]crop=4096:4096[cropped]; [cropped]scale=3840:2160[scaled]; [scaled][1:v][2:v]remap[remapped];[3:v]format=gray,scale=3840:2160,colorchannelmixer=rr=1:gg=1:bb=1[mask_rgb];

  [remapped][mask_rgb]blend=all_mode=multiply[out]

" -map "[out]" -map 0:a -c:v hevc_nvenc -preset p5 -cq 23 -rc vbr -maxrate 15M -bufsize 26M -c:a aac -b:a 128k short_w2.mp4

But unfortunately, the quality of the warped output is slightly lower than what is seen with OCVWarp - slightly less sharp. Perhaps changing the workflow to first remap and then resize (instead of resize to 3840x2160 first and then remap) might improve things.

Edit - 6 October 2025 - Yes - after doing the remap filter first, at the full resolution of the input video, and then the resizing to output resolution brings up the quality to as good or better than OCVWarp. ("Better" when OCVWarp's default avc1 video codec settings don't deliver as good quality as the hevc_nvenc settings above, I guess.) A separate repo has been created for this, incorporating a Python Tkinter UI,

https://github.com/hn-88/python-ffmpeg-warp/