FlexiCurve: Flexible Piecewise Curves Estimation for Photo Retouching


Chongyi Li     Chun-Le Guo      Shangchen Zhou      Qiming Ai      Ruicheng Feng      Chen Change Loy
S-Lab, Nanyang Technological University     
Nankai University    

Abstract


This paper presents a new method, called FlexiCurve, for photo retouching. Unlike most existing methods that perform image-to-image mapping, which requires expensive pixel-wise reconstruction, FlexiCurve takes an input image and estimates global curves to adjust the image. The adjustment curves are specially designed for performing piecewise mapping, taking nonlinear adjustment and differentiability into account. To cope with challenging and diverse properties in real-world photos, FlexiCurve is formulated to produce diverse estimations. The spatial dependencies among these estimations are implicitly modeled by a Transformer structure to improve local retouching of different regions. Thanks to the image-to-curve formulation, FlexiCurve only needs a lightweight network. Our method improves efficiency without compromising the retouching quality and losing details in the original image. The method is also appealing as it is not limited to paired training data, thus it can flexibly learn rich retouching styles from unpaired data. Extensive experiments demonstrate the efficiency, retouching performance, and flexibility of our method quantitatively and qualitatively.


Method

Visual examples by FlexiCurve

The first row represents the low-quality images sampled from Adobe 5K dataset. The second row represents the corresponding results by the proposed FlexiCurve, where (a) represents the results retouched by FlexiCurve trained with paired data and (b) represents the results retouched by FlexiCurve trained with unpaired data. FlexiCurve can deal with global tone and local properties well and does not introduce over-/under-enhancement regions regardless of paired or unpair

Overview of the FlexiCurve framework

The input image is first fed to the parameter estimation network (PE-Net) for estimating a set of knot points and curve parameters. The knot points and curve parameters define the piecewise nonlinear global curves (PNG Curves), which are used to adjust the level of the RGB channels of an input image. Multiple curves are generated simultaneously to provide us with multiple globally adjusted results ($G_{1}$, $G_{2}$, $G_{3}$). The final result is achieved by blending such intermediate results via a Transformer. (a) FlexiCurve trained with unpaired data, where global and local discriminators are used to distinguish whether the final image or randomly cropped patches are `real' or `fake', respectively. To preserve the original image content, a content loss is employed. (b) FlexiCurve trained with paired data, where the $\ell_{2}$ and $SSIM$ losses are employed for supervised training.

Results

Visual comparisons of the unsupervised models on MIT-Adobe5K-DPE dataset.
Visual comparisons of the supervised models trained on the same splits on the MIT-Adobe5K-DPE dataset.

Materials




Paper


Code

License

We retain all the copyrights of this method.

Citation

If you find our dataset and paper useful for your research, please consider citing our work: 

@inproceedings{FlexiCurve,
  title={Flexible Piecewise Curves Estimation for Photo Enhancement},
  author={Chongyi Li and Chun-Le Guo and  Qiming Ai  and  Shangchen Zhou and Ruicheng Feng and Chen Change Loy},
  booktitle={CVPRW},
  year={2023}
}

Contact

If you have any question, please contact us via lichongyi25@gmail.com.