CacheMeCode python option for include/input (instead of includegraphics) to work with tikzplotlib

[dflvunoooooo]

Is it possible to use the CacheMeCode environment to accept a .tex file instead of a pdf? I am trying to create a plot with python and the tikzplotlib library. Which produces a .tex file including a tikzpicture converted from the python plot.

This would be great, because with both python option, matplotlib export to pgf or tikzplotlib, will allow latex to set all text, which sets font, fontsize etc the same for any plot and the main text.

Here is a working python code. Working if run externally and included with input.

import matplotlib.pyplot as plt
import matplotlib
matplotlib.use('pgf')
year = [2014, 2015, 2016, 2017, 2018, 2019]
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=2)
plt.title("Simple plot $\\frac{\\alpha}{2}$")
plt.xlabel('Year')
plt.ylabel('Number of futurestud.io Tutorials')
import tikzplotlib
tikzplotlib.save("testbildMitTikzplotlib.tex") 

If I try to run this in CacheMeCode pytho and replaced the filename with __ROBEXT_OUTPUT_PDF__, it complains, that the file is not a pdf.

Edit: I tryed to create my own preset, like in the docs. And run the code with \cacheMe, but I get the error "(robExt) the placeholder does not exists" and a few others. Here is the code:

\runHereAndInPreambleOfCachedFiles{	
    \usepackage{tikz}									% Für Zeichnungen direkt in LaTeX
    \usepackage{pgfplots}
}
…
\robExtConfigure{
    new preset={matplotlib}{
        set compilation command={python "__ROBEXT_SOURCE_FILE__"},
        custom include command={%
            \evalPlaceholder{%
                \input{\robExtAddCachePathAndName{\robExtFinalHash.tex}}%
            }%
        },
    },
}

…

\begin{SetPlaceholderCode}{__TMP__}
    import matplotlib.pyplot as plt
    import matplotlib
    matplotlib.use('pgf')
    year = [2014, 2015, 2016, 2017, 2018, 2019]
    tutorial_count = [39, 117, 111, 110, 67, 29]
    plt.plot(year, tutorial_count, color="#6c3376", linewidth=6)
    plt.title("Simple plot $\\frac{\\alpha}{2}$")
    plt.xlabel('Year')
    plt.ylabel('Number of futurestud.io Tutorials')
    import tikzplotlib
    tikzplotlib.save('__ROBEXT_OUTPUT_PDF__')
\end{SetPlaceholderCode}

\begin{figure}
    \centering
    \cacheMe[matplotlib]{__TMP__}
    \caption{An example to show how code can be inserted}
\end{figure}

And also in an environtment without the placeholder. But it results in the same errors.

…
\begin{figure}
    \centering
    \begin{CacheMe}{matplotlib}
        import matplotlib.pyplot as plt
        import matplotlib
        matplotlib.use('pgf')
        year = [2014, 2015, 2016, 2017, 2018, 2019]
        tutorial_count = [39, 117, 111, 110, 67, 29]
        plt.plot(year, tutorial_count, color="#6c3376", linewidth=6)
        plt.title("Simple plot $\\frac{\\alpha}{2}$")
        plt.xlabel('Year')
        plt.ylabel('Number of futurestud.io Tutorials')
        import tikzplotlib
        tikzplotlib.save('__ROBEXT_OUTPUT_PDF__')
    \end{CacheMe}
    \caption{An example to show how code can be inserted into macros or environments that evaluate their}
\end{figure}

Edit2: There is a file created but it is only comprised of the python command python "robExt-5A490EBC95CB8773B752347B7CE4F90D.tex". The filename to run python on is the same as the file itself.

Edit3: Nor does the conversion to .pgf file from python matpltolib work. Here is a working python code which does not work inside cacheMeCode.

import matplotlib.pyplot as plt

plt.rcParams.update({
    "font.family": "serif",  # use serif/main font for text elements
    "text.usetex": True,     # use inline math for ticks
    "pgf.rcfonts": False,    # don't setup fonts from rc parameters, use Latex settings
})

year = [2014, 2015, 2016, 2017, 2018, 2019]
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=2)
plt.title("Simple plot $\\frac{\\alpha}{2}$")
plt.xlabel('Year')
plt.ylabel('Number of futurestud.io Tutorials')
plt.savefig("bla.pgf", backend='pgf') 

[tobiasBora]

Definitely yes, there is include command is input for that.

but I get the error "(robExt) the placeholder does not exists" and a few others. Here is the code:

This is because you forgot to specify a template (you get a weird error message), like template = {__TMP__}. (but you get a surprisingly not clear error message)

So I tried to write that but I can't manage to install the python lib so I can't test:

\documentclass[]{article}
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{robust-externalize}

\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEX_IMPORT__}
import matplotlib.pyplot as plt
import matplotlib
import tikzplotlib
matplotlib.use('pgf')
\end{PlaceholderFromCode}

\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEX_AFTER CODE USER__}
# .pdf is, by convention, the extension of the final file. Even if it is not strictly speaking a PDF file,
# it does not matter since \input does not care about extensions
tikzplotlib.save(get_filename_from_extension(".pdf"))
\end{PlaceholderFromCode}


\robExtConfigure{
  new preset={my matplotlib input}{
    python,
    add import={__MY_MATPLOTLIB_TEX_IMPORT__},
    % Code to run after the code of the user.
    include command is input,
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_MATPLOTLIB_TEX_AFTER CODE USER__},
  },
}
\begin{document}

\begin{figure}
  \centering
\begin{CacheMeCode}{my matplotlib input}
year = [2014, 2015, 2016, 2017, 2018, 2019]
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=2)
plt.title("Simple plot $\\frac{\\alpha}{2}$")
plt.xlabel('Year')
plt.ylabel('Number of futurestud.io Tutorials')
\end{CacheMeCode}
\caption{An example to show how code can be inserted}
\end{figure}

\end{document}

let me know if it works.

PS: next time, can you try to provide a full reproducible example, starting with \documentclass? It's much quicker for me to analyse as I can just copy/paste and see, otherwise I need to guess what package you loaded etc…

[tobiasBora]

(Ok, I can't install tikzplotlib because of nschloe/tikzplotlib#559)

[tobiasBora]

After adding

\usepackage{tikz}
\usepackage{pgfplots}

the above code (fixed now) works (needed to force matplotlib==3.6.2). Too bad tikzplotlib seems to be unmaintained as they do not want to fix nschloe/tikzplotlib#559

[dflvunoooooo]

This is because you forgot to specify a template (you get a weird error message), like template = {__TMP__}. (but you get a surprisingly not clear error message)

So I tried to write that but I can't manage to install the python lib so I can't test:

Awesome, thank you very much. I only missed the include command is input, option, sorry. So I have a working version with tikzplotlib, but mitplot does not work with you code. I kept all the python code in one place, because I often have to share it, and then its easer for others to understand. Here is a working full reproducible example ;) With tikzplotlib.

\documentclass{scrartcl} 

\usepackage{robust-externalize}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{pgf}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}

\robExtConfigure{
    new preset={my matplotlib input}{
        python,
        include command is input,
    },
}

\begin{document}
\begin{figure}
    \centering
    \begin{CacheMeCode}{my matplotlib input}
        import matplotlib.pyplot as plt
        import tikzplotlib
        year = [2014, 2015, 2016, 2017, 2018, 2019]
        tutorial_count = [39, 117, 111, 110, 67, 29]
        plt.plot(year, tutorial_count, color="#6c3376", linewidth=5)
        plt.title("Simple plot $\\frac{\\alpha}{2}$")
        plt.xlabel('Year')
        plt.ylabel('Number of futurestud.io Tutorials')
        tikzplotlib.save(get_filename_from_extension(".pdf"))
    \end{CacheMeCode}
    \caption{\(\mathrm{Number} \text{Number}\)Test Python tikkzplot mit CacheMeCode.}%
    \label{py:testTikzPlot}
\end{figure}

\end{document}

And the not working one with matplotlib. I suppose I can not simply add get_filename_from_extension(".pdf"). But why does it work with tikzplotlib?

\documentclass{scrartcl} 

\usepackage{robust-externalize}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{pgf}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}

\robExtConfigure{
    new preset={my matplotlib input}{
        python,
        include command is input,
    },
}

\begin{document}

\begin{figure}
    \centering
    \begin{CacheMeCode}{my matplotlib input}
        import matplotlib.pyplot as plt
        import matplotlib
        matplotlib.use('pgf')
        year = [2014, 2015, 2016, 2017, 2018, 2019]
        tutorial_count = [39, 117, 111, 110, 67, 29]
        plt.plot(year, tutorial_count, color="#6c3376", linewidth=2)
        plt.title("Simple plot $\\frac{\\alpha}{2}$")
        plt.xlabel('Year')
        plt.ylabel('Number of futurestud.io Tutorials')
        plt.savefig(get_filename_from_extension(".pdf"))
    \end{CacheMeCode}
    \caption{\(\mathrm{Number} \text{Number}\)Test Python matplot pgf mit CacheMeCode.}%
    \label{py:testMatplotPGF}
\end{figure}

\end{document}

After adding

\usepackage{tikz}
\usepackage{pgfplots}

the above code (fixed now) works (needed to force matplotlib==3.6.2). Too bad tikzplotlib seems to be unmaintained as they do not want to fix nschloe/tikzplotlib#559

Yes, matplotlib has to be lower than 3.8 to work with tikzplotlib. Very sad indeed, it seems to be a better library, since it does automatically detect the need for more space in the title, because of the fraction.

[tobiasBora]

I don't have a computer with me to test, but is the .pdf containing the latex code? If not it might be that matplotlib sees that the extension is not a tex and does something else in that case. Maybe use an extension like -to-input.tex (don't use .tex alone as it is already the source) and either move in python the file back to the pdf, or update the custom include command to include this other file instead. I will try to write an exemple when I have access to a computer

[dflvunoooooo]

No worry, there is no need for haste ;)
Ah, I didn't think it through. The matplotlib.use('pgf') option does not create a pdf but a .pgf file. I changed it to plt.savefig(get_filename_from_extension(".pgf")). Now it compiles without an error, but there is no figure in the document. The pgf file is created and when loaded manually, shows the correct plot. The corresponding log file is empty.

Edit:

I don't have a computer with me to test, but is the .pdf containing the latex code? If not it might be that matplotlib sees that the extension is not a tex and does something else in that case. Maybe use an extension like -to-input.tex (don't use .tex alone as it is already the source) and either move in python the file back to the pdf, or update the custom include command to include this other file instead. I will try to write an exemple when I have access to a computer

I am sorry. I grasp the idea what you propose, but I seem unable to do so. I am probably not familiar enough with robust extrenalize, sorry.

[tobiasBora]

To include a different file, you can add custom include command={\input{\robExtAddCachePathAndName{\robExtFinalHash.pgf}}},.

Demo:

\documentclass{scrartcl} 

\usepackage{robust-externalize}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{pgf}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}

\robExtConfigure{
    new preset={my matplotlib input}{
        python,
        custom include command={\input{\robExtAddCachePathAndName{\robExtFinalHash.pgf}}},
    },
}

\begin{document}

\begin{figure}
    \centering
    \begin{CacheMeCode}{my matplotlib input}
        import matplotlib.pyplot as plt
        import matplotlib
        matplotlib.use('pgf')
        year = [2014, 2015, 2016, 2017, 2018, 2019]
        tutorial_count = [39, 117, 111, 110, 67, 29]
        plt.plot(year, tutorial_count, color="#6c3376", linewidth=2)
        plt.title("Simple plot $\\frac{\\alpha}{2}$")
        plt.xlabel('Year')
        plt.ylabel('Number of futurestud.io Tutorials')
        print(get_filename_from_extension(".pgf"))
        plt.savefig(get_filename_from_extension(".pgf"))
    \end{CacheMeCode}
    \caption{\(\mathrm{Number} \text{Number}\)Test Python matplot pgf mit CacheMeCode.}%
    \label{py:testMatplotPGF}
\end{figure}

\end{document}

[dflvunoooooo]

Works perfectly, thank you very much again!

[dflvunoooooo]

Would it be possible to create a preset, which doesn't care for the filetype?

[dflvunoooooo]

Never mind, without tikzplot this is possibel with the normal python preset.

[tobiasBora]

What do you mean exactly? Creating a 100% extension agnostic preset is kind of hard, since there might be different kinds of inputs (like what do you do if you output multiple files?), languages, and include commands. But I can definitely try to find a more easy to use approach, like include command is input could be asked to check various extensions like first .pgf, then -to-input.tex, and finally .pdf. Would this be good enough for you? Another idea I have would be to run a mv XXX.abc XXX.pdf at the very end, of the compilation command but it would give an error if the file does not exist, so you would need different presets depending on the extension… another option is to run this directly from the python code (would be more efficient as well, but only specific to python).

[dflvunoooooo]

I mean, that with tikzplotlib there is a tex file created, and with the pgf option there is a pgf file created. It was not possible to use the same preset for tikzplot and pgf.

I would have expected for input to accept any file with the name, that is processible by latex. But you don't have to do that. I do can not use tikplotlib really, because it is doing funny things with numbers and I can not configure it.