[00:00:22] Bridget Scanlon: I'm pleased to welcome Martyn Clark to the podcast. And Martyn is a Professor of Hydrology and the Canada Research Chair in Environmental Prediction at the University of Calgary, and he's held this position since 2023. And prior to moving to Calgary, Martyn was a professor at the University of Saskatchewan and Associate Director for the Center for Hydrology at Canmore Coldwater Lab. Martyn also served as Editor-in-Chief of Water Resources Research journal from 2017 to 2020, and that is not a job for the faint-hearted, and I think it's getting even more complicated these days. And Martyn's research focuses on advancing environmental prediction, particularly how water moves through landscapes and how we can better forecast future water conditions.
His work is at the intersection of hydrology, climate science, numerical modeling, and earth system prediction. So it was nice to visit with you recently, Martyn, at the HydroML, Hydro Machine Learning, conference in Austin. And also we have interacted through CUAHSI on the landscape work.
So, that was very interesting. Thank you so much for joining me today.
[00:01:36] Martyn Clark: Well, thanks for having me, Bridget. It's nice to see you again.
[00:01:40] Bridget Scanlon: So, Martyn, the Hydro ML Conference was extremely interesting, and there was a lot of discussion about hydrology. And one of the panel discussions suggested maybe we should discard legacy process-based models or, so there's a lot of back and forth. Should we go fully AI ML and pretend like we know nothing or stick with our process-based models or somewhere in between, I think, is probably where we will end up.
But what are your thoughts on how we can use legacy models and maybe transition them to take advantage of all of the tools that are out there these days?
[00:02:16] Martyn Clark: Yeah, no, that's an interesting place to start actually. There's the panel discussion I think on the second day of the conference, and that got a little bit boisterous, I think we could say. Where there was discussion as to, okay, how are we going to move forward with legacy models?
And I think the problem that we've got is well, it goes back to the saying that historically every modeler has got their own model, and some have got two. And the models are based on similar code and concept. In many cases, there's more models in use than algorithms that are available to populate them.
So, for example, you could have the same evapotranspiration algorithm that's shared across multiple models. And a lot of the legacy models that we're using now do not really reflect our current scientific understanding of physical processes. So when we're thinking about that and we're thinking about how those legacy models have evolved over time, we've seen an ad hoc approach to model development where you can have a PhD student who does something and then a postdoc who does something.
Then a PhD student bolts something onto the side, and a postdoc bolts something onto the side of that. And the model is kind of cobbled together in haphazard ways. So that creates a lot of what we call technical debt that increases the cost of model development, maintenance, and applications. And it can be really difficult to use the monolithic structure of existing models to systematically isolate and evaluate a lot of the individual process parameterizations and add new modeling capabilities, et cetera.
So we're kind of at the point where we're at now, and then we've got the emergence of AI ML. And so I think we're at a really important juncture. So in my mind, we should be thinking about a clean sheet redesign for legacy models. We're not abandoning models. Of course, nobody's talking about that.
But what we're talking about is substantially improving the structure of the models so that they can be used more effectively in conjunction with machine learning. So we're talking about separating concerns, having a much better description of the governing equations that we have in our models.
Separating out how we solve those equations versus how we parameterize them, the processes within those equations. Thinking about composable components that we could use to build more complex models. But you know, that means that we kind of need a departure from the way that we've been doing things.
We can't just renovate the legacy models that we've got now. We need to think more carefully about the model design. Given what we know now, how could we move forward?
[00:04:53] Bridget Scanlon: Right. And it's all part of an evolutionary pathway, I feel, and those legacy models served a huge purpose for us to synthesize data, get data together. Kind of recognize some gaps that we had, a lot of gaps probably, and bring the data together and see if it makes sense and test different things.
But now we're at a different stage, and so a huge opportunity at the moment to advance modeling capabilities in hydrologic sciences. And Martyn, I really appreciate the effort that you and others at the CUAHSI program, Consortium of University for the Advancement of Hydrologic Sciences Inc.,
[00:05:33] Martyn Clark: Incorporated.
[00:05:34] Bridget Scanlon: Incorporated, sorry.
[00:05:36] Martyn Clark: Yes.
[00:05:38] Bridget Scanlon: On looking at landscapes across the North Americas and trying to understand the processes that are prevalent in different regions and you indicate that hydrology is fundamentally shaped by these landscapes and, how can we use the landscape structure then and incorporate them into the next generation models?
[00:06:00] Martyn Clark: Yeah, no, that's been one of the favorite projects of my career actually. It's been really good. So the National Water Center in the US had the challenge of building a national hydrological model across very, very large domains covering a wide range of different hydroclimates, wide range of different geologies a wide range of different regions where human water use is, quite different.
And they were interested in explicitly representing the dominant processes across different corners of the continent. Which is great as a vision. But us as hydrologists or the hydrological research community we've got a lot of information in individual regions and a lot of information on individual basins.
But we haven't really taken the time to provide a synthesis of that information and to be able to provide knowledge as to how hydrological processes can vary across large geographical domains. So that's what we did in the project, and I appreciate, Bridget, your engagement in the work.
You were one of the most enthusiastic participants, I'd have to say. But what we did is we broke, It was initially quite overwhelming for us, right? It's like, well, how do we do this? There's so much information. How do we at least begin to get a handle on it? So we decided to break the continent up into five regions.
And there was the northern region the western region, the central region, the eastern region, and the islands. And we had online workshops for each of those regions where we had invited speakers come and talk about what they saw as the dominant processes in each of those regions and to provide a starting point for the synthesis.
So we had a number of keynote presentations in each of those five virtual workshops and then some breakout discussions afterwards to catalog what the processes were. And then we've been iterating for the past couple of years on this, going back to the field scientists and thinking, "Okay, well, how can we provide the synthesis?"
And it's been coming together nicely, and we're now at the point where we've got the two draft papers complete. The key people in the team, so CUAHSI was terrific, Jordan Reed and his cadre of people were absolutely terrific in terms of, like, getting all of the engagement working.
The key people working on the project, Wouter Kibben did a lot of work on defining the hydrological landscapes, and Hilary McMillan did a lot of work to try to synthesize our perceptual understanding. And then Ying Fan Reinfelder she was just terrific as the great synthesizer of the group to try to pull all of these different threads together.
So we're quite happy where we're at now, but we're kind of at the cusp, right? Because we've provided the synthesis and now the next question is, well, how do we take what we've learned and think about how we can include processes and models.
[00:08:58] Bridget Scanlon: Right. Right. So I found many of the presentations extremely informative, and then at AGU you had the get together and stuff like that. It was really fun discussion and I learned a lot from listening to others. And even forcing myself to like, how can I synthesize what I've done, for 20, 30 years and what does it really mean?
How can we simulate this or incorporate it into models and stuff? And you mentioned during HydroML and during the synthesis work, that we're missing a lot of these processes in the current models. And one of the things that we talked about was the prairie potholes and the fill and spill aspects.
And then I like your term teflon streams, where we never allow the streams to lose water which we often see in semi-arid regions. And then, a big part of the Midwest has tile drainage, they're draining water off. And then of course the biggest elephant in the room of course is groundwater and, the short shift that it gets in many of the models.
[00:09:58] Martyn Clark: In some communities, yes.
[00:10:00] Bridget Scanlon: Right. So how how do you think we're going to incorporate some of these things into the next generation of models
[00:10:08] Martyn Clark: No, that's a very good point. You rattled through a lot of the processes that we identified as missing, When we're going through we can see that, like, a lot of the processes are ubiquitous. We've got the same processes occurring across many regions. We've got some form of, like, surface runoff generation.
We've got some form of, like, storage and transmission of water through soils, interception of water by the vegetation canopy, and snow accumulation and melt, even changes in discontinuous permafrost. We've got, like, some of those processes may be restricted more to some regions than others.
But in general, we've got a fairly complete description of the terrestrial water cycle, and that description is evident in diagrams like the USGS water cycle diagram, for example, because that in many respects provides a synthesis of our particular understanding of hydrological processes in general.
And then what you rattled through is, like, the tile drainage in the Midwest the losing streams in arid regions, the fill and spill of prairie potholes. That's one that is close to my heart, and I find that one really interesting because we've got dynamic changes in the contributing area of the catchment because the prairie potholes act as gatekeepers, and the area upstream of the prairie pothole does not contribute flow until the pothole is full.
So really interesting processes like that that are regionally specific and need to be incorporated. And then as you say, for a lot of the surface water models that have been applied over large domains we're missinggroundwater. So if we're thinking about the longer groundwater flow paths associated with mountain block recharge or in karstic terrain, et cetera, those are all missing from our models.
And as you mentioned, the teflon streams, where our modeling approach is to shunt water into the rivers and, let the water drain to the ocean without ever having any losses from the stream itself. A lot of those are unrealistic and need to be addressed.
But the other thing that we were trying to highlight, and I think this is really important, and this is where it intersects with the machine learning community, is how the landscape shapes hydrological processes. So if we're thinking about what equations to include in our models, the answer to that question is often quite obvious. We know what a lot of the equations are. But that's not the modeling challenge. The challenge is to quantify how the landscape structure shapes the storage, connectivity, release, residence time of water, et cetera. How the landscape shapes hydrological behavior. And that's not so much the equations in our models, that's more the parameters in the equation as to how do we quantify the storage and transmission properties of the landscape.
So when you view things through that, parameter estimation becomes a physics problem. It's not a statistical problem, It's to try to infer knowledge as to how the landscape controls hydrological behavior. And if we're thinking about processes on hand, process representation on the other hand, and parameter on the other hand I think, that's going to be how the machine learning community comes together with the hydrological process community moving forward.
[00:13:24] Bridget Scanlon: Right, right. That's very interesting. And, you mentioned the prairie potholes and Steve Loheide is doing a lecture series this past year and, talking about flooding in Wisconsin and he attributes to land use change in the late 1800s, the expansion of cultivation and changing from deep-rooted perennial vegetation to shallow-rooted crops and stuff.
So, understanding and developing an appropriate conceptual understanding of how the system is working and what the dynamics are is extremely important before you even start to model the system. And sometimes we have to test different conceptual understandings. And I remember when we were comparing some global models with GRACE data in parts of sub-Saharan Africa, we were seeing that the GRACE data was showing increase in storage but the models weren't and it's because they had the Teflon streams and these were endorheicbasins, that were internally draining, so the storage was building up and the models weren't allowing it. So we're always learning, I think, and, that's a positive aspect. I feel like we're always moving forward and we're learning different things all the time. So, you mentioned, the importance of parameterestimation and during HydroML you mentioned a large sample of emulators and how you're using those to simulate multiple basins at the same time.
Maybe you can describe that a little bit Martyn.
[00:14:48] Martyn Clark: Oh, yeah. No, exactly. And this is pulling from work that Andy Wood at the National Center for Atmospheric Research (NCAR) had been pushing forward, and we've been using that in a lot of our model applications activities.
The idea is that an emulator is really just a simplified data-driven model that we use to assist in the work that we're doing.
So the simplest case that we can think about it is that we're trying to calibrate a model for a single basin, right? And the calibration approach is that our model has got a lot of unknown parameters, and those parameters are unmeasurable at the scale that we're running the model at. And we often infer those parameters from differences between simulations and observations of stream flow.
So we characterize those differences between observed and simulated stream flow in terms of an objective function of model performance. It could be Nash-Sutcliffe efficiency, sum of squared errors or, something like that. And then we use that performance metric to infer what the parameters should be. So, where this started is adaptive surrogate model optimization, where you can see that you've got a number of parameter sets. And then we've got the objective function. So we're using the parameter sets to estimate the objective function surface. It could be done with a random forest. It could be done with a neural network, et cetera. But, we've got some data-driven model to predict the objective function surface. And then instead of doing a lot of parameter trials with the expensive model we use the data-driven model as a surrogate to estimate what the performance metric would be for a given parameter set.
And then that's effective because the surrogate model runs a lot faster so you can do a lot more function evaluations that you need for parameter estimation. So that's good. That's kind of like where the work started with the work from Wei Gong from Beijing Normal University about 10 years ago.
What Andy Wood has done which I think is really important, is that he's taken some of the ideas from Frédéric Kretschmer and others in the large sample hydrology machine learning community and applied them to this problem. So he does two things.
The first thing that he does is instead of trying to fit each basin individually, he fits all basins jointly. And when you're doing that, the emulator provides a fit to all basins simultaneously and can smooth out the idiosyncrasies that you have for the calibrated parameter sets in the individual basins.
The other thing that he does is he includes some geophysical attributes in the predictor matrix, so that you can estimate what the objective function would be in ungauged or poorly gauged areas and apply the emulator across the entire continent.
So Andy's been getting great results with that method. A lot of his papers have just come out. The key one to look at would be the Farahani et al paper that's in HESS that was published earlier this year. And we've been getting great success with that as well. And that's been the first approach that we've been using to take the model development work that we've been doing and provide applications across continental domains.
[00:18:02] Bridget Scanlon: That sounds fantastic. And so you have a detailed process-based model, you run it and you train the emulator with the input and output from the process-based model, and that runs 100 times faster than your process-based model and gives you flexibility then to test a lot of different parameters.
So that improves parameter estimation. And you mentioned Andy Wood looks at a number of basins at the same time. Because traditionally we would calibrate to a single basin, and then we would be trying to transfer to another basin, and oftentimes the parameters wouldn't be transferable or would be problematic.
[00:18:38] Martyn Clark: Well, that's been the big challenge, right? And it's been the Prediction in Ungaged Basins (PUB) effort, that's gone on for many decades. That's kind of been there in the background for most of both of our careers, I think. And we've seen that moving forward.
And the traditional approach is donor receiver, right? Where you've got a catchment that you're interested in, you call that the receiver catchment. And you're saying, "Well, what can be a catchment that we can use as a donor catchment to provide parameters for that receiver catchment?"
So we often look at catchments based on some similarity metric. If it be geographical similarity, in hydroclimate geology, et cetera. And say, "Well, we're just going to take parameters from the similar catchment and plonk them in this receiver catchment that we're interested in," And so we do that. But the problem is that when you calibrate for individual catchments, you're making the parameters to fit the idiosyncrasies of those individual catchments, and there's always problems with transferability. So what I think is the benefit of Andy's approach and the machine learning approaches in general, is that when you fit for multiple basins simultaneously, you're not tied to the idiosyncrasies of individual catchments, and the results are much more transferable.
[00:19:51] Bridget Scanlon: Right. And Martyn, so you spent some time at NCAR also, National Center for Atmospheric Research. So, and that's a great place to have part of your career in. You also have a framework where you can exchange different model components stuff. I think your title is SUMMA Structure for Unifying Multiple Modeling Alternatives.
Maybe you can describe that a little bit. And what drove you to develop that and how that has helped evaluate different types of models and stuff.
[00:20:19] Martyn Clark: Yeah, no, exactly. So this actually started with the work, gosh, it was like 20... Yeah, about 20 years ago. Where I developed a modeling framework called FUSE. It was the Framework for Understanding Structural Errors. So that was very simple conceptual bucket-style hydrological models.
And it was right at the beginning of my career where I was beginning to look at models and then try to understand their structural uncertainty. And then I was looking at the models and I thought, "Well, they're all kind of the same." Right? And like the equations have got like different form.
The people use different notations in the different papers. The models are described differently. But if you take a step back from the 10,000-foot view all of the models are the same. And I thought, well, we can put all of these models into a common template.
And so we can say, well, all of these conceptual rainfall runoff models or conceptual bucket style models have an unsaturated zone and a saturated zone. And they've got different buckets in each of those zones. Sometimes in the saturated zone they've got two parallel reservoirs.
Sometimes it's a single non-linear reservoir, et cetera. And they've all got the same fluxes. They're all calculating surface runoff. They're all calculating percolation. Some of them are calculating interflow, et cetera. But we've defined what we're now calling a master modeling template from which all of these individual models could be derived.
So what we did is that we created a new model and then used the parameterizations from a bunch of existing models in that common template. So that was useful for us because then we could say, "Well, what's the impact of the surface runoff parameterization?
What's the impact of the baseflow parameterization," et cetera. And we could do that in isolation, whereas, when we had existing models, there were too many differences between any two models that it was very difficult to attribute the performance differences to any specific modeling decision.
So we were thinking about this as a way to test hypotheses as part of model development, and that we were viewing that model was not as a hypothesis. Some people were saying, well, a model is a hypothesis. But we were more thinking that models are really an assemblage of hypotheses that are combined in different ways.
And what we wanted to do was to be able to have the capability to disaggregate a model into its constituent hypotheses so each could be evaluated in isolation. So that was 20 years ago. And then as we were getting to apply that we became increasingly disappointed. And it wasn't a problem with the way that the model was constructed, it was a problem with the breadth of processes that were included in the model. So we didn't have trees for example. We didn't have snow. And the complexity of the model was really lacking.
Then the next step was to build SUMMA. So SUMMA is both energy and water mass, and it's simulating the dominant thermodynamic and hydrological processes from the top of the vegetation canopy to what we called, the depth of active groundwater.
It's the surface hydrology representation that ignores a lot of the processes that we talked about before. Some shallow groundwater but, not groundwater the way that you'd think of it, Bridget. And then also with some flexibility in the spatial structure of the model so that we've had capabilities to simulate processes from summit to the sea, from headwater catchments to the coast.
So we've been moving forward in that. And one of the key things that we had as part of the design of the model is that we'd separated the concerns of numerical implementation in terms of the solving of the equations from the physical representations. So, the hydrologist would be responsible for identifying the process parameterizations, and the numerical analysts would be responsible for methods to advance those states through time.
So when we did do things in that way, it opened the door to the applied math community to come into our models, which they wouldn't have an open door, in many models because they didn't have that separation of concerns. And we've had a number of good collaborations with Raymond Spiteri at University of Saskatchewan as to how we can improve the numerical solvers in our models.
So a lot of the work with SUMMA over in the early days was to improve the numerical solution and then also to identify a number of test cases that we could use to check that all of the equations are implemented properly. And it's really just been in the last five years that we've been able to take what we've built and apply it over large domains
[00:25:01] Bridget Scanlon: Yeah, I think that's fascinating. We did an analysis many years ago where we were comparing I don't know, seven or eight different land surface models globally with the GRACE satellite data. And we said that the models are all over the map and the reviewers kept saying, "Well, you need to tell us why the models are all over the map and why they're not consistent with GRACE."
But I mean, as you say, that would be almost impossible because you have so many different things among the models, how they do different things, and so very difficult to compare them. So developing this framework then sounds like it really allows you to isolate, what is the main things causing differences.
And we also did inter-code comparisons and tried to isolate different things when we were doing a lot of unsaturated zone modeling. So, I think having this sort of framework then... I mean, is the National Water Model, is it moving in that direction to have this framework and then can exchange different models within it or use different components in it or,
[00:26:02] Martyn Clark: The National Water Model in the US has got a very similar design. So they're thinking about, like, how multiple models can work together in the same framework and how you might be able to use different process parameterizations across different corners of the continent and, things like that.
So, it's not the SUMMA code that we developed, that's not the National Water Model, but in terms of the philosophy shares many of the many common elements.
[00:26:27] Bridget Scanlon: Right, right. I think that's great. And then as you say, to isolate the numerical components from the hydrology components that allow improvements in both separately with different communities. During the HydroML conference, you also mentioned Gray Nearing’s paper, Water Resources Research paper where he questions the role of hydrologic science in the age of machine learning.
so I would like to hear your comments on that paper, and I think maybe you were trying to turn it on its end and turn it around for the conference
[00:26:58] Martyn Clark: Yes. Yeah, no, exactly. And for full disclosure and maybe all of your listeners already know this but Grey Nearing and I have worked together a lot in the past. So it looks like we're coming at this from different perspectives. But I've discussed all of this with Gray personally, and I think the two perspectives are coming together quite nicely.
So the key question is what role does hydrological science play in the age of machine learning? That was the title of his paper. It was published in Water Resources Research, I think in 2021, about five years ago, right?
And it caused a lot of debate within the community. And what Grey was doing is he challenged the hydrological community, and it's a terrific challenge to focus on developing a quantitative understanding of where and when hydrological process understanding is valuable, and a modeling discipline that's increasingly dominated by machine learning.
So it's effectively saying "What's the value of your science?" He was comparing... He had some comparisons in his paper, and he was comparing the skill of machine learning models, in this case the LSTM models applied across a large sample of catchments across the contiguous United States. He compared that against the operational version of the - model that was used by the River Forecast Centers, the Sacramento Snow 17 model, and then compared that against the current version of the National Water Model. And what he was saying was that machine learning models were way better than bucket models which were way better than the process-based models that formed the beginning of the National Water Model.
And he was quite bold, and he says that the reason why hydrologists have failed, I'm probably exaggerating his language, but read the paper yourself and make that judgment that hydrologists have failed because they lack theory. He said that we know this because we are able to provide much more accurate simulation catchments across large geographical domains, so that the theory exists but the hydrologists haven't been able to find it yet.
And our response... Well, my response, I don't want to necessarily bundle a bunch of other people into this critique was that we have the theory. We have the theory but the theory is scattered. Like, for us, theories are the stories that we tell to explain hydrological processes, right?
And we've got process explanations that are scattered across research watersheds worldwide. And people can come to conferences and explain how their system works. We've got syntheses of hydrological theory and, the initial synthesis of hydrological theory is that you can view the USGS water cycle diagram as a synthesis of hydrological theory.
And those syntheses of theory are included in our models. The motivation for the CUAHSI project supported by CIROH that we just talked about before was to provide a deeper synthesis of our theory to support a lot of the model development. So our view was that the theory wasn't lacking.
What we were lacking was effective ways to use big data to parameterize our models. And so this is the gap, and this is the opportunity where I think, what we're beginning to learn now is that using the tools from the machine learning community in a process-based framework can provide us with the interpretability of process-based models along with the accuracy of machine learning models.
And I think that's been the key accomplishment of Xiaopeng Shen's career. Well Chaopeng Shen's work over the past ten years. He's done a lot more than that, that's been an area that he's been focusing on.
[00:30:48] Bridget Scanlon: Well, it's nice that Gray put this paper forward and challenged the community. And you mentioned machine learning, long short-term memory, LSTM approaches and stuff. And, also in your discussion there, you emphasize... yeah, maybe you can fit the data better and get better performance, but you're lacking interpretability.
And then also maybe if you were dealing with other conditions other forcing outside of what you have trained your data model to, then maybe it wouldn't perform so well. But still it's really nice how you trying to emphasize then this parameter estimation aspects that maybe we need to up our game.
And in your program then you have been doing more and more AI ML and you mentioned that Xiaopeng and his differentiable modeling that he developed trying to combine machine learning with process-based modeling. Maybe you can describe that a little bit Martyn, and, how you can use that then to try to improve a parameter estimation using large data sets
[00:31:54] Martyn Clark: Yeah, no, exactly. And this is one of the things that I'm most excited about in the machine learning community. If we're juxtaposing Chaopeng’s work with the work on emulators that we talked about before emulators are always going to be constrained by emulator error, and the emulator error can be substantial.
What I like about Chaopeng's approach is that it's an emulator-free approach and can have a lot more applicability. So I can explain Chaopeng’s approach because maybe not all of your listeners are aware of it and maybe perhaps provide a little bit of history as to how Chaopeng’s approach fits within the broader literature. So if we go back to maybe 20 years or 30 years ago we're talking about the big challenge at the time is how do we calibrate spatially distributed hydrological models, And we talked about the inverse problem earlier, where we've got some performance metric and we're trying to use that performance metric to infer the model parameters. That's fine if you've got a lumped model where you've got a small set of model parameters. But if you've got model parameters that are distributed across multiple computational elements across a catchment, then the parameter dimensionality becomes enormous.
And you can't use a performance metric at a catchment outlet to infer thousands of parameters within a basin. Or at the time it was very difficult to do that using the methods that were available to us. So the approach that was used was parameter multipliers.
There was some type of spatial regularization strategy that is needed in order to make the problem tractable. So I think one of the first approaches of this is the work from Dave Tarboton with the TopNet model. There's a paper by Christina Bandaragoda in Journal of Hydrology in 2004 that describes this approach.
So effectively you come up with spatial fields of a priori parameters and you apply multipliers to those fields. So instead of like, trying to estimate the parameters individually in each catchment you've got an a priori field that you shift up and down and up and down and up and down and you're calibrating the multiplier, so. And then you've just got a small set of parameter multipliers that you're calibrating. So that was great, but it became very difficult to think about, well, how can parameter multipliers be transferred across different catchments because they're one degree of separation type of a thing.
It was kind of like difficult to use them. So then in like 2010 Luis Samaniego published a wonderful paper on multiscale parameter regionalization. So his approach was to say, "Well, we've already got transfer functions that we use to translate geophysical attributes to model parameters." You can think of pedotransfer functions where you're using sand, silt, clay to estimate the hydraulic properties of the soil, right?
You can think about transfer functions where you're using vegetation attributes to estimate the interception capacity of the vegetation, et cetera. So I said, "Well, we've already got these transfer functions, and those are the transfer functions that we're using to estimate the a priori parameter fields in the first place in the parameter multiplier approach.
So why instead of assigning parameter multipliers, why don't we just calibrate the coefficients and the transfer functions that are used to translate geophysical attributes to the model parameters?" And in doing that you had just a set of global coefficients that you could calibrate and instead of thinking about spatial regularization through parameter multipliers.
So that was very effective, and I think, that was a step change in the capabilities in the community at the time. But it was difficult because a lot of the transfer functions that we had were ad hoc, We had some pedotransfer functions, that were developed from limited soils data, but in some cases, we didn't have transfer functions at all. And this is where Chaopeng Shen's work comes in for parameter learning. So his approach is to say, "Well, let's think about a hybrid system where hybrid process-based model machine learning type system, where on the machine learning side we've got some type of data-driven model.
You can think of it as a neural network. Let's just call it a neural network for the sake of argument, and that neural network is translating geophysical attributes to model parameters. And that's applied across the entire model domain. That's to learn the relationship between geophysical attributes and model parameters.
And then the question is, well, how do you learn those relationships? You need some type of objective function of model performance. So that's where the process-based model comes in. So the neural network can provide you with parameters that you use in the process-based model to predict stream flow or whatever hydrological variable that you're interested in.
And then we need to use that information and the neural network to use the performance metric that we get from the process-based model to update the relationship between geophysical attributes and model parameters. So that's where that differentiable modeling comes in. So, what we need is the derivative in what we call the loss function.
That's the performance metric. The derivative of the loss function with respect to the model parameters. And if we can calculate that derivative and the loss function with respect to model parameters we can back propagate that loss parameter derivative through the neural network to update the weights of the neural network and thereby use the performance from the process-based model to infer the relationship between geophysical attributes and model parameters.
And, that's done across the domain as a whole. It's emulator free and the performance that I've been seeing out of that has been better than any other method that I've seen so far. That's my current snapshot and it's looking really good. So, we're in the midst of trying to apply this in our models, and we're in the early stages.
The way that Chaopeng and his colleagues have implemented this has been to recode their model and some machine learning toolkits like PyTorch or JAX. And that's, the way, a lot of people do that. That can be effective if your model is just a few lines of code.
So a lot of the applications have been for very simple bucket-style models but the approach that we're taking and there's others in the community who are pursuing this as well Jasper Vrugt and Jonathan Frame have got some nice papers coming out on this to calculate the loss parameter derivative analytically as part of the process-based model.
And then when we've got that loss parameter derivative, we can just back propagate it through the neural network in the same way that Chaopeng and his colleagues have done. So, we're excited about the possibilities. But you know, we're far away from getting any results
[00:38:42] Bridget Scanlon: Right. Well, it's great to have this new approach to parameterize these distributed models, that's a real challenge. And taking those attributes and improving the parameter estimation then would be very valuable. And you're going to do it with your emulator, and then Chaopeng does it emulator-free, so he doesn't include emulators.
But as you say, maybe some of the models that he's using are fairly simple bucket-type models or things like that, so
[00:39:11] Martyn Clark: Yeah. Well, actually we've got the emulator approach that we're using as a benchmark and when we're adopting Chaopeng's approach we won't be using that emulator approach at all anymore, and we'll be evaluating the extent to which Chaopeng's approach has advantages over the emulator approach
[00:39:28] Bridget Scanlon: Right, right. And when you mentioned the loss function, so for example, if you're simulating stream flow, is the loss function is the discrepancy between your simulated stream flow and the observed stream flow, right?
[00:39:39] Martyn Clark: Yes. Yeah, no, exactly. And then you would average... A typical approach is to average that loss across multiple catchments across your model domain.
[00:39:48] Bridget Scanlon: Right, right.
[00:39:50] Martyn Clark: So you've got a global loss function, which is propagating through the neural network which is providing a continental relationship or whatever the size your domain is between geophysical attributes and model parameters.
[00:40:03] Bridget Scanlon: Right, and one of the things that I noticed in your presentation for the Hydro ML conference, you mentioned quite frequently composable components and in building models and then how you can test those and evaluate the reliability. Maybe you can explain that a little bit, Martyn
[00:40:20] Martyn Clark: Yeah, no, exactly. So that goes back to where our model construction is, where we're trying to define a master modeling template from which all models can be derived and think about what the components are that can be used to build models. So the issue that we have is that people are putting more and more attention into modularizing their models.
But there's no standard way to do that. so people are modularizing the models in different ways where each module can have different scope of processes. So what I think the community needs is more of a Lego block approach where we're dividing standards for the model building blocks so that we can assemble the models, more deliberately and learn from each other more effectively in terms of what are the benefits of these alternative building blocks.
[00:41:10] Bridget Scanlon: Right.
[00:41:10] Martyn Clark: We've done that in a really simple way. So we say, well, okay, so we've got a bunch of conservation equations for energy and mass for a vertical column. So, our model is effectively a large collection of vertical columns across a continent, And each vertical column we've got conservation equations for different subdomains that we could have within that column.
So it could be, in the simple case, a subdomain for vegetation, a subdomain for snow, subdomain for soil, et cetera. And we've got conservation equations for each of those subdomains. And that we see as, the primary building blocks. And then the right-hand side of those conservation equations are calculated separately from the time integration of the states on the left-hand side, and that separates the physical representation from the numerical implementation so that we can begin to use third-party solvers, et cetera.
But that's the idea as to what are the composable components of our model and how can we define those. But associated with that also is testing. So how can we develop a battery of test cases that evaluate that the equations are implemented correctly each of the composable components of our models.
And I think that's going to become much more important as we're moving into agentic AI where we're getting AI assist in our coding to have, some tests, to make sure that work is done correctly, et cetera.
[00:42:36] Bridget Scanlon: Right, and some of the examples you gave were comparing against analytical solutions and things like that.
[00:42:41] Martyn Clark: Yeah. We had some were simple results that were presented in previous papers. Many of it was analytical solutions, We've got some really nice analytical solutions for snow actually, We've got analytical solutions for the liquid water content and vertical flux of water at different layers in the snowpack.
And then we also use some lab experiments. So we had some lab experiments for cryosuction where you can get migration of water towards the freezing front to make sure that the behavior is represented appropriately in our models.
[00:43:17] Bridget Scanlon: I really liked the diagram you showed at the conference and I'll include it in the highlights for this podcast on the web where you show the evolution of land modeling since the '70s. so that's nice. And getting more and more complex and considering more atmospheric forcing and linkages and more recently, I think we're trying to bring in human intervention and human water use, and it gets really...it can get quite complicated. And another aspect Martyn, is with this agentic AI and AI assistant and everything, we can gin up code in our sleep.
[00:43:50] Martyn Clark: Right.
[00:43:52] Bridget Scanlon: So, it seems like a recipe for potential disaster in the future, with everybody ginning up all these things and, how do you think we can try to coordinate or, avoid a lot of confusion and everything with a lot of stuff going out and, have some control on what's happening?
[00:44:11] Martyn Clark: Yeah, so, I think this is where we need some leadership in the community. The way that things are moving forward is just, we're kind of being overtaken by the capabilities. What I see, Bridget, is a huge asymmetry. So, agentic AI is helping us with software creation enormously, so what we're seeing now is that an individual researcher can create software at a scale that used to require an enormous software development team. And so agentic AI has advanced software creation enormously. But it's not helping so much with testing, validation, reproducibility, governance, maintenance, sustainability, et cetera. So we can see software being created that could be difficult to sustain. Software being created that can be difficult to maintain. Software that's being created that can be difficult to verify because all of those tasks don't scale nearly as well as software creation, or they're not accelerated at the same rate. So we need to be thinking really carefully about where we're putting our effort and that we're not getting ahead of our skis in creating software that ultimately does not have utility.
So, the way that I've been talking about this with colleagues has been integrity debt, is that rapid pace of software creation is causing debt in all of those areas that I just mentioned. And we need to be thinking about the creation of that debt and how we can sustain and pay that debt in the new age that we've got moving forward.
Sorry, go ahead.
[00:45:56] Bridget Scanlon: Oh no, Go, ahead if you have something else to say related to... I was going to move on to the publishing aspect.
[00:46:03] Martyn Clark: Well, it's the this isn't necessarily separable from the publisher aspect also, right? Because just as agentic AI can help you create software, so too can large language models help you write papers. So...
[00:46:21] Bridget Scanlon: And review papers.
[00:46:22] Martyn Clark: And review papers. So, yeah, they're not they're not necessarily that separable.
[00:46:28] Bridget Scanlon: so, I mean, I think are we seeing a big increase in the number of papers that are coming out now and, very difficult to find reviewers and how would you describe that? I mean, did you see it during your tenure as chief editor? There was the increasing number of papers coming in and, difficulty of finding reviewers.
And then, you would always say in the past, "Oh, it's peer reviewed." Well, what does peer review mean these days? It was reviewed by AI or...
[00:46:55] Martyn Clark: Yeah. No, exactly.
[00:46:56] Bridget Scanlon: So how do you see the publishing... and when we chatted before, you mentioned a conference in San Francisco many years ago and, trying to highlight some of the publishing issues.
[00:47:10] Martyn Clark: Yeah, no, exactly. so when I was editor-in-chief, I finished my term at the end of 2020. So, this is really before, we've been dealing with agentic AI. And it was then hard to find reviewers. It was hard to find good reviewers. That's always been a challenge in the publishing landscape.
But at that time AGU was really pushing FAIR, findable, accessible, interoperable reproducible, the FAIR software standards. And think the journals did really well with the A in FAIR, making things accessible, but, and perhaps findable, but not necessarily interoperable or reproducible.
So, a lot of work was being developed, but the papers were reviewed, but the code was not necessarily reviewed. And I think that needed to happen in order to advance the scientific progress that was being made. The current trajectory that I think we're on is that we've got massive increases in our capabilities.
But we're using those massive increases in our capabilities to pursue our current trajectory. So what it can mean is that we're writing 10 times more papers on the same topics, and we haven't yet pivoted to where I think we need to pivot to, well, what are the new questions that we should be addressing that we haven't been able to address before?
[00:48:25] Bridget Scanlon: And I think part of this is driven by the young people and the performance metrics that we use
[00:48:31] Martyn Clark: Yes, exactly
[00:48:33] Bridget Scanlon: We can count but we can't read. I think that's what you mentioned.
[00:48:38] Martyn Clark: Yes.
[00:48:38] Bridget Scanlon: You're lucky if you can get a co-author to read the paper you're putting together, and so really we need to reevaluate. And I can't blame the young people for what they're doing because that's the metric that's been used. But you mentioned the San Francisco DORA, and I forget what DORA stood for but, you know, that– I thought that was interesting. I downloaded and looked it up and stuff, and I thought it was an interesting concept, trying to look at other metrics beyond just number of publications and citations and things like that.
[00:49:08] Martyn Clark: Yeah. Yeah, no, exactly. And this DORA's been around for a while. It stands for the Declaration on Research Assessment. And we mentioned this. When I began my term as chief editor, I wrote a critical paper on the citation impact of hydrology journals. I mentioned DORA at the end there. And that's 2017 paper in Water Resources Research.
And there we were arguing that we needed to move away from outputs and move towards impact. And so outputs would be, well, how many papers have you published each year? And that's what you said is that managers know how to count, but they don't know how to read type of mentality, And more towards what's the outcome of your work? What's the importance of your work? What has your work enabled, et cetera? What are you known for? And so that's encapsulated very well in DORA. And, there's been a lot of signatories for DORA. And it's effectively moving more from a quantitative approach to research assessment to more of a qualitative approach to research assessment.
Focus more on research outcomes than research outputs.
[00:50:15] Bridget Scanlon: Right, and maybe it's a bit more difficult and that's why we're hesitant to go that direction. And DORA also included, discussed it, in terms of institutions and organizations and journals and all of that sort of thing. So they were trying to address all aspects of the community, not just the researchers.
And I thought that was a nice aspect. So, Martyn you, teach hydrology in Calgary– at the University of Calgary. So what do you tell your students about what to do in terms of using AI ML? I mean, I gather in the past I've heard that some institutions, maybe British Geological Survey or others said that they weren't allowing their researchers, I'm not sure that's still the case to use AI ML.
So I'm wondering what advice you give to your students going forward
[00:51:01] Martyn Clark: Embrace it. Use it. But use it carefully. If you don't use AI ML, you're not going to be competitive in your career, I'm kind of struck by some comments that were made to me by a director of a very large research enterprise a couple of years ago. And these comments were made right at the time when AI numerical weather prediction systems had overtaken the skill of traditional numerical weather prediction systems.
And this director, he said that he's got two new rules for his staff. And the first rule is that you must be machine learning literate. You don't have to be a machine learning expert, but you need to know what machine learning can do for you, You don't have to use it, but you need to understand what it can do.
And second is that all of the software development, you can't code by yourself anymore, right? You need to code in partnership with AI coding tools. And the key is the partnership, Bridget, I think. Is that, we're not asking AI to do our jobs for us. We're working in partnership with AI to help improve the products of what we're doing.
So I've had a very difficult assignment that I had in previous years where I've asked people... My, the first assignment of my process-based modeling class was that the students had to code a model from scratch without being given a single line of code, and then evaluate the impact of different numerical methods on the errors, from the model simulations.
And, now that can be done with the support of agentic AI, so, I encourage the students to use AI support, AI assist in the work that they're doing. But I've also got a section in the assignment where they need to explain how they use the AI and how they check that the results were credible
[00:52:56] Bridget Scanlon: You mentioned something and I cannot remember the phrases you used Martyn, you said yeah, I won't take your job, but I, I can't remember what...
[00:53:04] Martyn Clark: Oh, well, it's an old saying now, I think, is that AI won't take your job, but someone using AI will.
[00:53:10] Bridget Scanlon: Right, right. Yeah. And if listeners hadn't noticed Martyn is not Canadian by birth, from New Zealand, and you're doing some flood work there, and unfortunately we don't have time to get into it, but a lot of flooding issues in New Zealand in these past couple of years so, and you're getting back to some of that again also.
So our guest today was Martyn Clark. He's a professor of hydrology and Canada research chair at the University of Calgary. Thank you so much, Martyn. And we will put the script and also highlights of the podcast and linkages to papers and some figures on the website so people can get some background information also
[00:53:49] Martyn Clark: Yep, that's awesome, thanks for having me, Bridget
