Closing discussion, 26. January 2016, recording and transscript

Present were (in order of appearance):

ARJ – Alexander Refsum Jensenius
JS – Jan Schacher
GTP – Germán Toro Pérez
AT – Atau Tanaka
KC – Kathleen Coessens
DB – Daniel Bisig
AS – Angela Stoecklin
PN – Patrick Neff
HJ – Hanna Järveläinen
AN – Alex Nowitz
CS – Christian Strinning

(Julie Harboe had left, but her comments are referenced ).

Recording of the project discussion, duration is 1:07:52.Download the mp3-soundfile for offline listening here (size 139.2MB, right click, then “Save Link As..”)Download the .pdf of the transcript for offline reading here (19 pages)

Forthcoming: a digest of key-points regarding project management, goals, and further activities.

 

00:03 ARJ: I guess, to repeat myself from last year. I guess perhaps, but then I think your idea is right in terms of perhaps looking in from a daily perspective, because in a way it makes sense to have Cadoz’ categories when you talk about some producing actions, because they are, in a way, well defined. As you know, I disagree with the way he uses the gesture concept, but I think those two categories there are fine in a way. I think they can also easily be used to be extended to…

If you cannot extend beyond the analog/digital divide, what we have been working on is to… What we are using right now in our group is talking about five different layers. That covers the analog and digital divide.

You have embodied, which is the first one, which is where you produce sound only in your body, like a singer, for example.

Then you have the tool-based, where you have directly in contact with a surface and instruments, so for example playing a drum in your hands. That’s the next layer.

Then you have the mechanical, where you’re playing the piano, where there is two layers in between, or more. You’re not in direct contact with the excitation object. Then you have the electronic, which is maybe the analog electronic, like circuit boards type of things.

Then you have the truly digital, which is also electronic, of course.

At first, at least that has helped in trying to categorize these things, because you can more precisely define what actually happens and it can go in between them. You can find the excitation, modification, selection elements for all of these. You can make up a matrix to categorize this. I can send you some examples of how we use this.

Then the other question is about the gesture.

[laughter]

01:55 ARJ: To me that’s something totally different, because that is not related to this at all. I mean, sound projection itself is very physical. It’s biomechanical/mechanical. To me, when you talk about gesture that’s really about the meanings and the intentions involved in this. That’s a different scale. It’s a different layer, really.

You can talk about the gesture coming out of all of this, but just pressing a key on a piano is to me is not not a gesture, it’s an action. It can be interpreted as a gesture if you like, but the meaning that that has may be the same meaning that you may produce with your voice, in a way.

You can say, “Goodbye.” You wave goodbye in many different ways — with your hand, your foot, whatever. The meaning is the same. The action is totally different. In a way I would encourage you to perhaps, as you said, turn around a little bit when you look at these different perspectives.

02:55 JS: For me, that’s absolutely evident that there is these layers. I like the five layers you just mentioned, because those are kind of mediations, and still instrumentally related mediations. In dance, actually, instrumentality is not there in the same way, so we’re on a different level anyways.

The terms that we now heard from Angela, like the focus, or the quality — qualities was one word also that we heard from Bertha last year, thinking in terms of movement qualities. That’s on a different level, which is not to say that as musicians, we don’t do that.

I think it’s just one domain which is less described for music, because we’re always more busy with the more instrumental sphere. I think those two coexist in both levels. Well, the question is, the body an instrument in dance? I don’t want to get into that debate, but it’s a big question.

03:58 GTP: From our point of view, this question on the level of meaning is what comes on the third place, when we see it from the perspective of the audience. That is, let’s say, the third. It’s a different perspective. There, of course, the semantic layer plays an imminent role, a very important role for us.

Let’s say there were two questions, and with those questions, we want you to open, define discussion about the piece before going to other discussion, and that was based on two things. The question of visibility, that has been addressed many times at some point ANgela was saying also, “my goal is not to make fixed choreography, but to make visible the direction.”

The question is for me also, in Vortex, if we are being aware of the second player, let’s say Jan, and of the third player, the machine learning, how…I was putting this question yesterday to Beat, if he’s aware to differentiate what you do and what machine learning is doing, because this is a question I’m putting to me.

I’m unable to differentiate this. I can answer it for me not. not clearly, so I would like to have this kind of polyphony of those three players in a more clear way.

Of course, saying this, I’m taking my personal view of that as a part of the audience, and the question of the visibility is one. The second is, of course, what kind of aesthetic experience of those pieces are allowing us? What kind of articulation or different kind of sound, movement articulation, and what this generates in terms of a third thing that has a new meaning.

In that sense, for me personally, I come back to the concept of polyphony. When I try to focus on that on the different parts of Double Vortex and of Moving Music, where I grasp or I’m confronted with this kind of polyphony, and at the level of hearing or seeing is the point where more meaning is being generated. Where I’m able to read it from different points of views, where let’s say a different quality that I don’t find in music, that I don’t find in the dance. I’m not being an expert dance auditor, but…

06:50 AT: You’re talking about a polyphony of meaning?

06:55 GTP: Yeah, a polyphony of formal layers that allow me to make different readings. This the aesthetic potential, from my experience as part of the audience, I would propose here. But just to open it, to open the discussion.

[pause]

07:24 JS: My first thought would be what I mentioned yesterday already, is that a complete piece or work of art is always highly dimensional. There’s always many, many, many dimensions, and that would be my answer already. That _is_ already polyphonic in that sense, but I think it’s not exactly the way you mean polyphonic.

I agree that then perceiving these different layers, and actually also, over time of… So from the audience perspective, but also me being my own audience when I perform, going through these different levels of polyphony is like shifting through: realizing, “Oh, there’s another relationship. There’s another layer. Oh, there’s another change in how things are.”

That might generate what you call an aesthetic appreciation, or a meaning generation. I mean, signification maybe more than meaning.

08:23 GTP:: Yeah, I know.

08:25 JS: Sometimes, it’s really signification in terms of a sign. For me, this multi-layered, and actually, I think there’s an almost infinite number of layers. But we’re only capable of focusing into certain ones, depending on maybe our culture.

08:46 GTP: Our tuning.

08:48 JS: Our bias, our focus, so when you try to read what is the live electronics doing, is because you have this focus, and you have a conceptual grasp. Somebody who’s not informed, then looks at space and gesture, like Julie just said.

09:08 GTP: I would argue that everybody has a basic curiosity to understand what’s going on, so that’s part of the game. We were discussing that also after your intermediate showing at the special of visibility. I thought yesterday in the version, your dance you performed yesterday, there was something about, let’s say the gestures you were using, you were reducing.

We talked real good about that, so I had the feeling that for the audience, it was even more visible because you had kind of motivic elements that were being used many times, so you were able to understand what they generate, the different variations of that.

They appear in the third part, too, and that made the piece more transparent. I’m not saying one thing is better than the other, but I’m just saying there’s a different aesthetic quality.

10:15 AS: That was actually part of the evolvement that we took, also as a result of that first showing, which we realized it’s already so complex in itself. How can we help the public be able to read the piece, and not necessarily my language, or the music or the interaction language, but what the piece is about, where it comes together?

On that, we decided to reduce the first part to make it almost didactical, so the readability would actually become more transparent, and more and more evident also why there is this connection. Because it could otherwise be just another dance piece.

10:59 AT: I think the legibility and the visibility of the technologies are, anyway, going to be ongoing. There’s no one answer. For me, anyway, I was really impressed and touched by the concert, how discrete and visible and acoustic it was.

But I come back to this question of Kathleen, if I may, because the slide is still up, and because this is the question of the interaction. Be it visible or not, the other perspectives of the analysis of the structure, of the piece, or the audience reception to semantics, anyway, would happen with any piece of music.

Given that we’re in this setup of technology interaction, yes, the grid is one way to look at it. Kathleen’s right, is that it’s not just a way to look at it, but it’s probably the way it’s constructed, whether you like the words or not.

But the way of Angela’s exploration…

12:22 KC: And experiential…

12:23 AT: Yeah, experiential exploration of the body, the self, the space, is a different process. I’m going to make a jump here, where maybe there’s some hope that this, to come back to machine learning…

[laughter]

12:45 AT: I’m just trying to focus on the relevant things, anyway, for me, or for what I’m able to speak to. Maybe there’s some hope that the machine learning can get us out of this grid, can give us a form of human technology interaction that doesn’t have to be mapped out in this way.

However, the contradiction is that actually, with the machine learning, you have to be much more structured in when to turn the recognition on. Which zone are you looking at to segment the gesture? We dive into this. Many of us, I think, we share this. Machine learning already as a term is very romantic.

[laughter]

13:32 AT: We dive into it thinking, “Well, it will just learn. [laughs] Now we don’t have to worry about 180 degrees here, 45 degrees there.” The disappointment is that actually, to do anything with it, you have to almost be more determined.

Maybe, just to come back to this, it should offer a kind of potential of this experiential exploration. But to configure it to do that is something we haven’t really talked about, or what the world’s struggling with. It’s a real struggle.

14:07 KC: Yeah, I put the word struggle.

[laughter]

14:12 AT: Good, so I finished on a word that you have noted.

[laughter]

14:15 JS: That is also our intention of making it its own agent, so that at least perceptually when we perform, it’s not under our control and determined in that sense. Of course, that’s a kind of fine line.

14:36 AT: Yeah, but if you want to make a multi-agent system, why wouldn’t you use dynamic time-warping?

14:41 JS: No, of course. This is using very simple models to try out this. Of course, I was just thinking, when you say we have to determine more and more and more, I think there will be a level when you determine so much that it gets out of your grasp again. Then it might, again, get this quality of autonomy or independence. I don’t know.

15:05 KC: What is interesting is that at one side, you have this human learning structure, which you can put it in a very easy classification. At the other side, and that’s where you want to come with technology, is that you want in certain instances to come up to the complexity of an ecosystem. And that in the work that was presented yesterday, that figures you confront in an artistic way, such a system, with the complexity of human experience and human artistry, it merges.

That is why I said, “Well, we need also to look from the other side,” because it’s much more than that. The complexity of human experience and sensoriality comes in.

In fact, that is why it is not possible to do twice the same presentation. Because _that_ does twice the same thing, but the human complexity, and the human experience and movements, and life, and organic, is impossible to reproduce a second time.

16:23 JS: Maybe that’s because the layers that are determined might stay the same, but the way the layering polyphony occurs in each presentation is actually different, in different weights, in subtle ways, and that changes the focus.

16:43 KC: That is interesting, in which way machine learning, or machine, in fact, becomes also very complex, because it interacts with that complexity.

16:56 AT: Maybe if it’s a computer vision system. If it’s Microsoft Kinect, and you take it to another concert hall, and the space is not as black as this black box, the lighting condition…

[laughs]

AT: No, these are actually all the technical problems. Then if you do a training set in your black box theater and take it on tour, your training set’s not going to work. Then it will make a mistake, but is the mistake to be considered an agent? I just used…

KC: Yeah, but then you have architectural conditions that change.

17:26 JS: We had this yesterday going from our lab to the concert hall, and the vision system reacted differently, and that was one of your feedbacks. (pointing to AS) It felt different. Some things didn’t work the same way, and that’s exactly what happens, is that it’s not fixed. It’s not stable in that sense.

17:43 AT: If I can be a bit provocative, this is what Apple and Google are doing for you anyway. If you want to really deal with machine learning and deep learning and the ecosystem…You say ecosystem, we imagine all green and leaves and all that.

17:58 KC: No, not at all.

18:00 AT: But ecosystem is this, “Hello, what do you want? Oh, you want to take a taxi to the airport? Oh, your Uber will be ready on time.”

18:05 KC: And ecosystem tries to play with your complexity, because when you come on some books, and then send someone else, and afterwards, they’d offer you some real…

[crosstalk]

AT: That’s right. Then there’s some real stupid complex…Yeah. Something that you already bought.

[crosstalk]

AT: Yeah. “You’ve been shopping for slippers. Don’t you want slippers?” “No I bought the slippers already,” but then…

[laughter]

AT: It’s not to make light of it, but this is all this area of machine learning, that we think is rigid, and we try to be more fluid with it. However, that ecosystem is existing in a very, very ad hoc, but very determined way. Very, very determined way, but that’s extremely robust in responding to the very aleatoric nature of our walking to different airports, to walking around different cities, being geo-localized. They’re doing it.

19:03 JS: Of course.

19:06 DB: I wonder if the way that a lot of machine learning algorithms work is actually because there is a desire to have the system come up with something like this. It’s a very clear, Euclidean grid-like subdivision of categories.

AT: Categorizations, yeah.

DB: When we were interested in a more ambiguous quality, or fluent quality, then we would actually need to look for different types of machine learning.

19:35 AT: I think that the answer to the agent question is, it’s not that, “Oh, wow. Dynamic time warping’s made a mistake,” or, “That’s cute,” or, “It’s going to be like a person you play with,” but no.

Maybe we are looking for outputs for the system, or uses of machine learning that are more fluid, and that are very different to the classifications, and the typically brittle outputs of machine learning systems.

19:58 JS: What I always think is that we actually should look at differentials as a general term, because that’s actually where the fluidity of elements is. So not just to classify, look at one level, but actually look at relationship between things.

20:12 AT: But why don’t you use regression more?

JS No, I could. It’s a…

AT: Or some in-between zone? That’s, I think when you get technical, what GVF ought to…

JS: Of course.

AT: But then, GVF is very brittle in the kind of input that it’s demanding in order for it to be fluid.

20:36 JS: Absolutely. To me, this poses the question of the tool set you use, the concept you use. For me, it’s also like zooming in and zooming out, in terms of, “What am I doing?” It’s a technical process. It’s a conceptual process. It’s an artistic process. There’s all these layers.

I think the focus also shifts in time, in terms of the length of a process. We start off with a focus on one end, and then we say, “OK, let’s put the focus on it.” To me, this is cyclical.

Of course, in one of our next cycles, we need to go exactly now, as [inaudible 21:26] , “OK. Now, what’s the next step?” Then it’s good to have clarity of what it implies.

21:31 ARJ: Two more confusing elements for me, I guess. [laughs] Just following up on the machine learning thing is…I guess in the classic artificial intelligence world, you have always had the division between the rule-based systems and the machine learning type of systems.

In a way, trying to make a rule-based system, or describing what happened, while I guess the machine learning is a bit more chaotic because it’s not rule-based in that way. It’s super wise, but still, it’s trying to figure out its own way of doing this. I guess that are two different things, fundamental way of different things, trying to build the rules from scratch, or to just figure out what’s there.

From an artistic point of view, I guess what you really want to get that, because a true gesture recognition system, where you recognize the idea] and not the technical stuff. That was also what I proposed when I proposed the GDIF format.

Which never became anything, but still I think is a good conceptual idea, where I tried to separate out different layers. The gesture layer is kind of the top layer, where you’re only talking about meanings and not ideas. Then you have all the technical stuff at the bottom.

Through different layers along the way, ideally, you should be able to create a system where you can do the proper gesture recognition when you are in a setting. For example, in a dance, music setting, where, “Now, let’s have our machine recognize and record the gestural ideas that we have, the meanings that we want to convey to the audience.”

Then the machines will figure out the rest themselves, so if you move the setup from your lab to the concert hall, or even out in the streets, that shouldn’t matter. Because if the system worked perfectly, then the ideas, the meanings would be able to move around.

Of course, it’s not like that in the real world. I still think that ideal is an interesting thing…

[crosstalk]

DB: Is this ever achievable?

ARJ: No, of course not. I guess, or perhaps Google will solve it sometime.

[crosstalk]

DB: There’s so many horizontal influences. It’s not just a hierarchical abstraction on…

[crosstalk]

ARJ: Sure, but still, the idea of working that way, I find kind of interesting.

23:55 JS: What I realize is it’s extremely context-dependent. By context-dependent, I mean you might have one configuration, and then you take the same configuration and you alter one element.

For example, we add another type of sensing to the dance piece. Then my meta thing actually is questionable, whether I can link the meta thing back to the technical thing, or whether I have to go through the whole process again.

Ultimately, I think the system needs a lot of world knowledge and social knowledge, because the meanings are actually embodied social and cultural, etc. So we have these two domains that are, to me, quite far apart still.

24:45 DB: I would be very happy if the system’s able to look for contrast, so regardless in which setting you do the performance, the system learns to differentiate between salient things and non-salient things.

So it’s sort of the task to perform, or, in the very beginning, for the performance, or as a preparation for the performance, to make this distinction clear so the system doesn’t have to have the context.

25:05 JS: But then you still don’t get the gesturality. The thing is, I think when we talk of gesture as a semantic layer, or meaning or signification, that implies more than just…

25:17 DB: Even that, the semantic layer has a certain component to it, so it’s not that the system has to have a knowledge of semantics, or that’s my hope, because you’re right. It just becomes an expert system for me too. [laughs]

25:30 JS: Let me ask you this. Angela, or also what Kathleen said, and from Aline in Antwerp, when they talk of states or of qualities, how would you approach these terms from this kind of lower-level measurement analysis point of view? How would you do that?

DB: Well, first of all, by moving away from categorization. [laughs]

AS: Or using others, I think you would have to use other parameters somehow.

DB: But we have to differentiate within categories.

JS: Also, parameterization I think is already difficult, in a way.

AS: These are very metrical somehow, and I think maybe one would have to go away from them, more to temperature, muscular density, things like that maybe, or quickness of input-output in the brain. Things like that, maybe, whatever you’d have to change the parameters that you measure. I don’t know.

26:39 AT: Then the problem of performance comes. If it becomes experiential, you create an interactive system. It responds to your state longer term, interaction, temperature and everything, Does that then become legible to the audience, who is sitting in a very standard proscenium stage…

[crosstalk]

27:00 AS: I ask myself sometimes, is the legibility really what you want to achieve? Is that not something which is interesting for the people behind it?

Is it not that you’re trying to get to a specific aesthetics or a specific experience in that performance, which is different than if you didn’t have that, if you just had a normal concert with normal music and normal performance interacting?

For me, that eventually is what is more important. And yes, omehow, you have to lead the people, maybe, into it, because they know there is these two things to make them be able to connect them. But eventually, I don’t think you want them to, what we talked about, understand what is behind it.

27:50 GTP: Seeing how the pieces evolve through the time, they move from a more experimental situation, more focused on phenomena, towards what we call a piece, which are more condensed, a set of elements that you can recognize.

This question of recognition is, I think, not only for the machine’s importance, or for the audience importance, just to reset your mind map in the moment where you are preceding something, and it’s a question of choice.

I would say that Moving Music kept more this experimental setup, whereas Double Vortex developed stronger towards what we call the composition, with a more defined, formal, with transitions which were very clearly tried out and so on.

Then I see two different qualities also for me from the audience. When I see you are looking for, you are in search modus or experimental modus, I tune differently when I hear, listen the same thing. It’s different to hear a concert of composed music or to go to a venue of improvisation. I’m just thinking differently.

Then there is, at the end, an aesthetic choice. I want to keep this experimental quality, or I want to generate a piece that can be, as such, recognized?

29:41 JS: In the large term, the question of what the goal is in terms of using these techniques, using these tools, using these instruments, of course comes down to aesthetic choices, but also to interest in relationships.

To me, what Julie was saying, she finds it fascinating even though she’s repulsed by it in a way. This contradiction I find, that, for me, is a motivation, for example.

Or the idea that using artificial intelligence processes will give us something, and that’s a romantic notion, and then corporate America is using it for nefarious purposes, or profit. That opens up a field of tension in regards to these tools, where I think then the motivation should actually inform the concept of how we work, and what we want to achieve.

30:43 AT: It’s too bad Julie’s not here now, but what she brought I think was very interesting, to say, “OK, from a historical perspective, OK, I don’t care about the technology.” But then she said, “Oh, this is the bubble.”

It’s too easy to then start complaining about corporate America. Because actually, we’re dealing with your music, one concert, a couple pieces, and we’re focused entirely on this. If we had other pieces from other composers that are using similar technology, and other approaches, then we could compare them or talk about them.

Right now, since we’re completely focused in this lab setting, we instantly go to the big picture. Is it gesture? Is it meaning?

It’s an amazing bubble that you’ve created, because you are multitasking, and have multiple instruments and multiple pieces that you presented last night, and you are our host. You’re running this project where you have psychologists, music theorists and external advisors come in and then we talk about these that you’ve created. It must be a huge pressure, but it’s incredible luxury in some way.

With anything that we’re doing and we have many common interest and knowledge to share, but the focus is entirely on these works. I wonder, what are we talking about really? Are we talking about how you can make a better piece? How pieces have changed since the last iteration?

[laughter]

32:44 AT: Whereas, if Julie came into a concert… It was a public concert last night, so the concert was an interesting point, border line, because we had been in the same room all day.

Suddenly it became a public space. It was not quite a public concert. It wasn’t advertised if it was an art exhibition it was a solo show of Jan Schacher, but in a lab setting. There was Cathy van Eck, very interesting. Good that she was there. We didn’t talk about it.

If Julie came into a concert where there was a Bach Partita and then there was a dance piece and a Cathy van Eck piece and someone else’s piece — maybe a Jean-Claude Risset piece. Then maybe we’re contextualizing in a comparative way that will give us something to talk about actually.

30:55 JS : I’m totally with you on that, myopic. That’s in a way, the consequence of a research culture, which demands that you clearly cut out…

AT: Of course and I know this very well, I’m part of the problem.

[laughter]

JS: I was hoping you would bring some solutions…

[crosstalk]

AT: We’re trying to be part of the solution, aren’t we?

GTP: Yeah.

AT: This is why our research is part of the solution, supposedly.

34:24 KC: At the same time, even if in this lab the context is not there by way of all these people who come from all different horizons, we bring our context with us. A lot of us know different artists who also create interactions between acoustic and digital elements.

34:50 AT There are enough…

KC: The bubble is full of holes.

AT: It’s not a shared knowledge. Yeah, that’s right. The bubble is full of holes. Everyone brings their own baggage that’s implied, we understand some other references, but they’re not explicit.

35:08 GTP: I think we are putting the pieces on the center. Are we asking if this gives a common ground for those different perspectives? The slide with the triangle was shown many times. Still, it’s the overarching structure and that’s one of the main hypothesis of the project is that this to focus on concrete pieces.

Because when we’re talking about terminology we’re talking about different cases in a general format focusing on specific pieces. Would it help us to understand what is meaning of gesture for instance, for the generation of musical forum?

So these were the questions we are putting, specifically you have all the questions from machine learning. Technically you have all the different questions there.

I think that’s a quite interesting question right now. This model that Jan developed, and it makes sense and so bring to help us to understand a little bit more what meaning of gesture for the different interests is or not.

36:30 JS: My hope is that it can be generalized, but I think there’s always limits to how generally it can be right. You’re right about talking the disparity between everything and the singularity. I’m very well aware that each performance is a singularity.

The project design tried to address this by creating these blending spaces.

37:00 AT: You do a great job. This group that you’ve assembled, it’s amazing. I’m very jealous and I’m impressed, but what does it teach us? Does it teach us to create the trombone piece? The diagram shows a very complex series of interactions for one piece.

Is the goal to try to even generalize? Should it be generalized?

Or if we’re to find a diversity of approaches from the same set of problems, could there be then, in that triangle diagram in the middle of this arts practice — you know, right now it’s the practice of one artist who happens to be the PI of the project — could you imagine expanding that center core in a series of commissions or making the technology available as a platform so that students could try, and then see what comes out?

Then you’re letting it go into the ecosystem of creation.

38:12 KC: Now, because you were not here last year.

AT: Yeah.

KC: What I would say, and I think when it comes to music, the difficulty was you were here long.

What is quite important is that it started, so now it’s the third session, I was there. It started very open. This is the one year that it really focused upon the works of Jan and really the whole analysis by all the..

I remember the first year it was totally open. The composers were talking about, how can we find the discourse about gesture, about corporeality? How can we do that?

The same with the technologists and the same with… I remember you were really coming from outside and saying, “Oh, I love to do this, but how will I do this? I come from a whole totally different sector.” It was very open.

They invited Donnatienne (Michel-Dansac) at that moment. It was a singer.

39:20 AT: I couldn’t come but I remember seeing the program. I thought, “Whoa, it was the opposite of Jan.” I said, “What is Jan doing there?”

KC: It started from a very vague zone in which this happens. The second time it was with Berta Bermudez. It was all about movement, about how the semiotic theater, the movement, the gesture, the sensorial.

I want to put it into brackets that we focus really. I think because now it closes a bit, it close until it’s truly focused. It started really from a much more open way.

40:02 GTP: Meeting Donnatienne helped us a lot, because we were talking about something very specific. There was a piece, there was a practice, there was a certain experience so everybody could make his thoughts, so confront his own thoughts with a very specific situation and at least talk about the same piece, which was good.

Then when we moved to Lachenmann, that was again very good, because it was another field of experience that allowed to develop discourse from different perspectives.

I think that if you invite external composers or you open it to students, what we have been doing in other projects, or you do it with a personal approach as we did here, working on specific pieces helped us, at least, to concentrate.

Because if not you end up talking about something too general. At the end, for me one of the questions here is, “What is the meaning of gesture in a composition context?” I learn more seeing it on specific situation where I can see all the difficulties, all the different qualities, all the perspectives than keeping it on a general way.

The question is, how diversity of works, what kind of diversity is needed in order to confront different views. It’s a good idea to have it more related to the interest of a single artist or if it is better to broaden and…

[crosstalk]

41:52 AT: Maybe there’s a shape, that it did start broad and now we’re at the point focused and then the future of the project will be it goes out again.

When it goes out, then the question is then all this work has to be somehow transferred to people who aren’t in this room, or to people who didn’t follow the whole journey. How will this knowledge be encapsulated and written down for someone?

Because if you give this to another composer, then do they have to start all over this whole process? If they pick up the technology, do they have to make all the same mistakes? We would hope that this project then gives something. Then the question is, in what way is that knowledge…

[crosstalk]

42:40 JS: That’s definitely what we’re at this point going is the last section, what we’re going to do, is how to really collect and make that available.

In a way also for ourselves, become more clear what we’ve achieved, exactly what you’re saying, so that we can then leave the bubble or extend the bubble, because of course it’s always through that.

Those thoughts have already informed one submission we’re making for now and then spring for another project, we’re exactly pursing that model where we can get multiple perspectives and then do this comparative or differential view.

Then of course it’s in what domain are we active, socially speaking? Are we in art music, is it club music, is it music consumption, is it sub culture? I know that’s a topic that is in your group is also present.

That’s one perspective I’m also interested in. The question of what can it bring to the composer, we can then pose what does it bring to the dance or design? Also, “What does it bring to a listener in the tram with the headphones?” It’s nice that you mentioned the game world, which of course is also a way of sound work which is contemporary, essentially.

The general and the specific have to be balanced. It’s an accordion. We’re open and close it continuously. I’m very happy that you see this clearly at this point to make the accordion closed and then we take another breath, so to speak.

44:40 GTP: There’s one point I want to make and I must apologize because I have to leave in two minutes. One thing is, if I see that concentrating on specific works help us and that I think was a good idea.

The question that remains open is how those three different layers are really communicating with each other. That’s something we discussed for one year. We try to find ways to come together, but still every track was following its own path.

This crossing of the tracks, music technology, music analysis and psychology were not really coming together. Or it’s the question, why the product is still not finished and we’re still possible to try to match to come a little bit together.

This was another assumption of the project. I think we have to think about how this, in which cases it was really happening and why it was not happening in other cases in order to see if this methodology makes sense for other projects or not.

46:00 AT: I’m surprised. So you didn’t do any measurement the last night of the audience? There was no questionnaire, there was no GSO sensor on everyone’s fingertip. There was no pressure sensor on the seat, buttons.

[laughter]

JS: Yes. Those thoughts have been in the air, but it’s a question of scope.

AT: Sure.

46:27 JS: In terms of succeeding and failing in some aspects there’s also blending of these different disciplines, the question is how do we evaluate whether it’s successful or not. Because of course you can only overlap to a certain point, because the disciplines have specificities of methods and questioning and also terminology.

Then also the question is, “What is the process of making those overlaps visible?” Because of course the main task being in a specific discipline, you occupy the discipline, you’re not occupied with the blending of the space.

That’s something I’m going to be thinking about for the next few months when we’re collect theories, collage theories, is going to be exactly that. How do we make the blend visible?

47:32 DB: That can be very problematic. How do you benefit from the results from other tracks?

JS: Yeah, for example. I think also juxtaposing materials can already be enough to make it visible. We’re commentary or just saying, “We take the work that Hanna has been doing an empirical way in just maybe one or two really important points.

Can we juxtapose the effort thing with the dance music. Can we use that knowledge…

[crosstalk]

48:04 AT: Or use that knowledge to compose your next piece for example or make it available somehow implicit in the technology, the wrapper around your machine learning algorithms so that some composer can use it and just use an effort slider.

[laughter]

48:20 JS : For example, and I think in a less structured way that already happens, but I think it needs to be shown and made explicit and to be useful. Also by making explicit and fixing it and putting it in a accessible place it becomes a tool.

Because right now it’s insights, experiences, thoughts and that is all really important, but it’s not to at the state where it can be a tool yet. Or where it can become somebody else’s starting point, which is what you’re saying and I completely agree.

Maybe to close the discussion, and we had a little bit of contact over dinner last night. One perspective I’m really interesting in is the network idea.

AT: Is what?

49:10 JS: The network idea. You mentioned an idea of having a gesture symposium where little groups meet. That would be one aspect that I think we should pursue and draw in resources.

In the same sense to the tool development level. You were asking about if it’s available and then we can make more efforts also to really make this position of the people. Then also I’m interested in seeing your work and whether that connects.

To me, this is very concrete and something I’m coming back to you in the next week or two, or three weeks of keeping those threads active for future opportunities. I understand will also be interesting to you.

AT: Yes, of course.

50:05 JS: It doesn’t have to be a formal collaboration, but more kind of pooling our minds and our insights. This is a specific thing, so then also the artistic research side, I think I will try to also have something. The JAR submission is exactly this intention of generating an artifact, secondary art work which carries that.

I don’t know whether you’ve seen our website I have started making since last summer. (to CS) Can you plug in and show MGM? You’d have to plug in the projector.

This is something I’m also quite concerned about, is really putting materials online and the traces that we have, especially media traces, but also we’ve published a bit and we’ll published more. I think we’ll have 20 publications that when we’re done for two years, in different fields.

51:19 ARJ: That’s as Atau (Tanaka )was saying. You have been publishing here and there. Really I think if it’s a web page or something else but really to have some way of condensing the whole project, would be really valuable.

As far as I know this project, is really the most overarching one on music, and as I know about at least. We have several projects in line with them. Also when I worked there were things going on but not with the same scope as you have here.

In that sense it’s quite unique and it’s an important story to tell and also to really document it in away so that it’s easily accessible and manageable for everyone really, being published by the project. I guess this web page might be exactly that.

52:12 JS: The intention of this is to be the online presence of the project and that’s where I’m going to pour in effort in the next couple of months. This is really the distillation of all the efforts. If you may have noticed that we had a film team again last night. We’ll have all eight pieces online, for example, that will be one thing.

If you go to the tool section (to CS), if you go down there’s also the second tools also sketched out — that one (showing the motion capture visualization). For example, and this is very sketchy still, because I haven’t…This comes out of the publications, in the work.

(to CS) If you go the work section, just go to works itself and roll down. I would really like to have, like you say, an overview. Then so publications of course, if you click on publications, that’s still growing. That’s half of it, more or less.

I think that’s really important to make that available. Of course, publishing in our context makes it visible. Maybe a joint activity might also make it visible, so we’ll have to talk about it concretely now, how to organize and do something like that. I’m very happy to invest energy in that, from my side.

54:00 JS: Because ultimately the question is, “What remains? What’s left over when you finish an arc?” Of course we start the next one, but somehow collecting all of that. So I hope the JAR can actually be like another page, or another side of that.

AN: Sure.

54:23 JS: That’s why I was really thinking of it, because how do you say, different pages, or different surfaces?

54:44 ARJ: It would be really useful also to have. I’m not sure if this is possible, you might have thought about it, but if you can condense the entire project into one, what’s your finding, in a way, or doing a model. I there something that we can say that, “This is what we managed to get to”?

Not necessarily in a scientific sense, but in a more…because that’s when you really get to the knowledge part, from you’re gathering everything, it’s a lot of material and you’re condensing and condensing and condensing.

55:20 JS: Yeah, and that’s what I mean by making it accessible, in a sense drawing out the essential elements and keeping them side by side. I think, there will be something like findings, and that might just be a half a page or a page of major ideas, but I was thinking of how to, not evaluate, but validate what we’ve done.

On the individual sections I can imagine how to do that. In the overall thing, don’t know yet. I don’t know whether they have any experience for this kind of…

56:03 ARJ: It’s tricky. This is just a brainstorming idea, but I think it would be really super valuable, for everyone else at least, and also hopefully for you too, to really try to look at how the different parts of your diagram influence each other, and really try to come up with… Because then, I think that could lead to some interesting findings.

What exactly did the music psychology and music analysis teams get out of each other, in a way? What’s there? Is it possible to summarize that? What particularly is the interesting thing, for example, between the artistic work and the technological developments? Then you can hopefully get into the center on the triangle in the end.

56:55 JS: That’s the methodology thinking that Atau was mentioning yesterday. I don’t know whether you’ve seen my MOCO paper from last year. I’m already active in these kind of, not super-high level, but a bit meta just to see to see whether it’s possible.

Because in the long term, that’s how we progress. It’s not in the concrete work, that’s short term. The long term is, “What is the method, really?”

I hope to actually try to do what I did with the motion capture and other practices and to try to do this with the triangle. I don’t think it will be a sound bite, in the sense that you can have one sentence that’s like, “That’s what we learned.”

[laughter]

57:42 JS: “We learned that gesture is…” I don’t think it’s that easy. Maybe we should hire a PR firm to kind of generate that.

[laughter]

57:55 JS: Then underneath, you have all of these smaller things.

I would encourage, if after today you have some insights, to send them to me, because that would also be super useful. Thank you for your feedback, because you had some keywords in there that are…

In a sense, it might actually help me to condensate it more, if I have this outside view. At the moment, I’m too close to the thing, so in a sense, it’s good to have the wider circle also.

58:33 AS: That actually invites me, from the artistic point, to also think about what’s happened here now, and how it feeds back onto my process that I made, on the actual process, or how the interviews fit that and things like that.

58:50 JS: Like you said, those are valuable assets in terms of methodology. Of course, the publication thing, some of them are really formatted for scientific publications. Some of them are more essayistic. I’ll do a lot more essayistic stuff also for my dissertation this year, so you have these different zoom levels.

59:23 PN: Maybe you’ll also a final word from me Let’s not forget the very specific use case of gestural-controlled electronic music, which is actually missing instruments. We approached it through a mere mimical expression, through instrumental gestures, so that at the end, we really focus on this very specific situation.

JS: Missing instruments?

PN: Missing instruments, yes, or looking at your very own piece. (references ‘new islands’)

JS: That wasn’t totally in the focus, but it’s been running on the sideline, of course. Or kind of underneath, this has been running for a while.

AT: How much time remains in the project?

JS: Only a couple of months, but that doesn’t mean that we will… I mean, I will spend more time finishing the documentation things and writing it up, etc.

I think the concrete work is one. Maybe we can do experiment seven with the motion capture of the trombone piece. To me, that poses the question, “What’s the next project, and how do I get the resources, and what’s the submission, etc.?” That’s a longer-term thing.

The other question is, “What are the intermediate steps?” I didn’t want to submit the project while I was running the project to get kind of a back-to-back thing, because that’s quite disruptive.

We could have applied for an extension of a year, I think. I’m not sure we would have gotten it, but that also would have been quite disruptive in terms of workflow, etc.

1:01:12 AT: What was the second one?

JS: The second one was the SMC paper we wrote last year.

AT: OK, that’s SMC.

JS: That was kind of where I was looking at the mixed-methods research, which is the qualitative-quantitative thing from the social sciences, and trying to apply it to this.

AT: But is SMC interested in social science approach to me making music?

JS: No, of course. It was showing kind of the project layout. Again, it’s a methodological questioning, in that sense.

1:01:58 AT: But it’s really interesting. Again, I’m provoking, and try and go quick, because again, we’re at 12:15. There’s a deadline. Here, we have NINE, SMC, ICMC, MOCO, but this incredible rich multidisciplinary group that you’ve got here, why do you stay in this ghetto?

[laughter]

AT: It’s our ghetto, but I just ask you this question.

JS: Well, what’s outside the ghetto? I know there is other…

AT: Nice neighborhoods, parks, green spaces.

[laughter]

AT: I don’t know. In my own work, I’ve been pushing more CHI than NINE, but just because if we’re saying this might have meaning elsewhere, and this is the question of, “OK, is it to make your piece, or is it to make for us, or to make better pieces?”

But if we’re learning something about the psychological perception of F, or if we’re learning about dancers’ exploration of the self and the body and space, that could be for MOCO. Yes, but I know there is an elsewhere.

JS: Of course, there is.

AT: To identify who it could be…

[crosstalk]

JS: For example, we submitted to, what was his name?

DB: Oh, yeah, Metamusical Creation.

[crosstalk]

JS: The whole algorithm generative thing, where you had a journal article that was rejected there, on subjectivity, for example.

AT: I think it will be nice for us to have a collection not of the publications that are made, but the rejections, actually.

[laughter]

AT: This is an interesting idea, a website of not academia.net, but rejections.net.

[laughter]

AT: This could be good, because that’s where some of the best ideas are.

KC: Or rejections of abstracts for chapters.

AT: That’s why I keep all my old drafts, because then when you have something, you go and pick something that wasn’t published.

KC: The strange thing is that sometimes, you are too early with that. Two years after, you re-ask if it’s interesting, and then suddenly, they find it interesting.

AT: By the time it’s published, you’ve had to make the reviewers happy. You had to fit it in the page limit, be rigorous, so all the good stuff is gone.

[laughter]

JS: Yeah, the other thing is self-publishing, which is in a way and then promoting. We’re institutional, we have an institutional presence, and of course we can put stuff up. But then, you know.

But you’re right, I’m constantly asking myself, “What is the appropriate context?” Then also the question is, “How much effort do I put in the neighbor context where the chances of being accepted is really slim.

1:04:35 AT: There’s a trend in science now, where people are publishing early results, and there’s blog-style feedback, right? Could we do something like this in our field?

ARJ: ArXiv.org?

AT: Yeah, at the same time that I say this I’m horrified but the energy, the in real-time input that it would have to have to respond to read, and to respond and to discuss in a forum-like setting, but maybe that’s one way to continue this very rich conversation you’ve started.

Expanded out, expand the community. You know we would immediately hopefully participate because we’ve been here, so we know what you’re talking about. But then if it’s online, then it opens up, becomes of public expression.

1:05:27 JS: Should we put a forum on this and then kind of get started?

AT: Well, yeah. Forums are a nightmare, yeah.

JS: Yeah, that’s my notion.

AT: To look at, what are the good models that are working in scientific publication where some of these incremental public…? I don’t know. Who’s got the example of this?

1:05:45 ARJ: Plos One? I think is we had one Plos One article now, which I think is actually quite interesting. It’s a very different way of doing the whole process for reviewing, etc.

[crosstalk]

ARJ: Everything is accessible. Everything needs to be. That everything needs to be published there, all the support materials, etc.

AT: Frontiers is the sort of thing that…

AT: Is that what they’re after?

1:06:11 ARJ: An then empirical musicology review is also interesting since they have this open peer review system as well. ICMPC would have been an interesting thing for us I guess, for the psychology part.

1:06:22 JS: Yeah. That’s for example, one neighboring field where I’m a bit apprehensive of going in because of the dogmatic aspects that you can…I perceive, when often music in psychology like the Oxford Club.

[crosstalk]

AT: Well maybe you don’t have to be first author every time, you know? Then push Hanna in the area where psychology is more…

[crosstalk]

HJ: …is dead.

JS: I’m already pushing her.

[laughter]

AT: I mean, depending on the target venue, then your team gets re-shuffled in who carries that paper, who is first author and that.

JS: Absolutely.

ARJ: In terms of my participation as well.

1:07:08 JS: Well, they would be informed by that but it’s also a different company. It’s a more general question, it’s not just now for this but more general because of course I think scientific writing is just one thing. We’re not just dealing with scientific writing. For example, for me, media content is super important. It’s not just words but it’s also other traces.

If you can’t transport the actual thing, you can transport it in as many different ways as possible, So the research catalog in general is interesting as kind of a surface. You just plunk down your materials and then you have a first thing.