Too big project? (exclamation marks & fatal errors)

Absolute Pyramid lover here. I have 150k great things to say about this machine. Today’s post unfortunately is a cry for help. I am aware support is deprecated… I have the latest OS installed.

I’m working on an ambitious project (~15 tracks with 20 patterns each to run through 20 sequences). Time signature is 3/4, track length 18. Fun stuff.
Until I got a frozen screen, when I (accidentally?) tried to rotate all notes within a pattern. We are talking 4 note-chords at most, but there are ~50 of these in a pattern. I panicked a bit and shut down the device.

After another couple of days I suddenly got an Error message with a couple of number codes, which I can’t remember (I thought this error would reappear, but it never did… and I don’t believe those are logged in the device or SD Card afaik). 5 min later, I got a Fatal Error (frightening experience). Since then I work in fail-soft mode, with a lot of care, I do minimal moves, I don’t use the rotate function anymore, I copy/paste small amount of notes etc. workflow is degraded but progress is possible.

Today I started getting Fatal Errors erratically when I would change Sequence Number. Thought it was time for an SD formatting…

I have formatted my 16Go SD card (long formatting), reinstalled the OS, and put back my project in there. No Fatal Errors yet, but memory is playing new tricks…

Sometimes I get an exclamation mark (out of memory, if I’m not mistaken) upon loading the project, sometimes I don’t.
Copying a track is simply not responding. Copying a pattern of a track is not responding neither. Copying all notes within the pattern of a track sometimes works, sometimes doesn’t, and sometimes gives me another exclamation mark.
Sometimes I can copy successfully, but it’s when I paste that I get the exclamation mark (only some notes are pasted). What is weird is that I have managed to reproduce scoping down to getting an exclamation mark for trying to copy a simple, individual, 4-note chord.

Two questions:

  1. Is my project too big / does anyone work with the Pyramid on even bigger projects?

  2. Is it possible to have a corrupted project ? Not SD Card, not device, but actual data in the project? (I’d like to know if I should prepare a backup plan to finish my project from my laptop for instance)

I don’t know how to help you, but 8 days without out a reply from Squarp is messed up. I hope you sent a ticket or email. Good luck!

1 Like

That does sound like an ambitious project!

If the 15 tracks have 20 patterns each and the tracks and patterns are all 18 bars long then, I think that would mean, each bar would have to have less than 2.4 MIDI events in it on average to keep within the stated limit (or less than an average of 43.3 events in a pattern). So I think it is quite likely that you may have reached that limit.

Some of the inconsistences you describe do seem a bit odd. I have no idea how the memory in Pyramid works, though. Perhaps there’s some allocated (reserved?) RAM for the copy buffer, perhaps other RAM can be used if it’s available but maybe that depends on something like how fragmented it all is.

I’m afraid I have no idea whether it is possible to have a corrupted project.

2 Likes

Hi @camzeech,

You say each track in your project has 20 patterns, and that these are all 18 bars long? Is that the case for each and every track? Obviously, i don’t know what your project is like but I can’t help wondering if you really need a new pattern for every track for each sequence change or whether all really need to be 18 bars long.

I have found myself tending towards longer track/patterns (maybe a bit like you) as I’ve got to know Pyramid (and the sorts of things I want it to do) a little better. This puts less strain on the limited number of ‘sequences’ you have with Pyramid, and changing them at less frequent intervals with regular sized (and longer) patterns means you can get some complex (and possibly evolving) arrangements going on. But, unfortunately, it might not be all that efficient with respects to the number of MIDI events your project will have to store in its relatively small memory. It might be that you end up with a fair amount of duplication of MIDI events across your patterns because you want some (small?) variations at some specific places in your project and there aren’t really enough (Pyramid) sequences available to use a sequence for every small variation you might want to make. So it might seem that you are a bit stuck if you want to make long complex arrangements with your Pyramid.

But I don’t think that is necessarily so. It might be that all the tracks in your project are ever evolving and that there is next to no repetition of any bars or phrases. If that is so there may be little that can be done. But presuming there is at least some repettion of bars or phrases in your project I think there probably are ways to reduce the amount of memory you are using with this project. Although that really will depend on what is going on in your project and where you want it to go.

A couple of rhetorical questions: Could you have made some of the (later) sequences you have there from any of the patterns that already existed in other sequences? Are there some patterns in a sequence that you have that could be made by switching from one sequence to another at some point within the 18 bars? You have 12 more sequences available and they are relatively memory cheap. So, for instance, you could create a new sequence to change one (or more) patterns over at the appropriate point during the 18 bars. Whether this would work will, of course, depend on what run modes (and possibly other settings) you are using (it will also mean another sequence and a little more messy chain of sequences). Thinking about that, playing with when sequences change and using patterns with different run modes can be a good way to create some variation from largely pre-existing sequences. But I suppose that would be of more use at an earlier stage than you are at currently with this project in question. It requires creating new patterns (if you need to keep the old ones with the previous run mode), though if I understand correctly from your description of your project, your existing patterns would behave the same whether they were in relatch mode or free mode because it seems your sequence changes come at regular intervals, every 18 bars and the patterns are (all?) 18 bars long (which would probably mean the trigger mode would also behave the same way).

Personally, when I have a few relatively long sequences arranged I tend to start looking for ways to create another (probably of the same length) by looking for places to switch between the sequences that I already have. I might have to create a few new patterns for some tracks to get the results I want, so I will probably be creating another sequence or two as well, but I’m unlikley to need to create new patterns for all the tracks. So maybe I might get another half minute or so arranged at relatively little cost (memory-wise), and I’ll have more sequences available to play with to help create ‘another’ future sequence. And who knows? I may need to create fewer new patterns to get something I like this time. It’s sort of like the opposite of diminishing returns, maybe. Of course, sometimes I’ll need to make something more like a completely new sequence with nearly a full set of new patterns, it really does depend. But I do like to try and recycle. But that’s me and one way I like to work with my Pyramid. What works for me might not for other people. One of the good things about Pyramid (and probably many other sequencers) is that it allows for many different approaches. However there are constraints, and Pyramid’s style of arranging (sequence mode) and its fairly small amount of memory are probably the most restrictive of these when it comes to making long, complex arrangements (at least those that have a fixed and repeatable order). The challenge, of course, is to find ways of doing things that allow you to get the sort of results you want within those constraints.

I must admit that I am looking at what you have said and seeing the words, ‘~15 tracks with 20 patterns each to run through 20 sequences’, and thinking that that sounds a bit likely to be a bit memory intensive and I can’t help thinking that maybe you may have some (unnecessarily) duplicated patterns amongst all that lot. I may have misunderstood or taken what you said (that would be the ‘each’, I suppose) too literally and I may be way off mark with what I’m saying and what I’m about to say. Apologies if that’s the case.

I wonder if perhaps you use the auto pattern feature, and keep it on (set to ‘at new seq’ in the misc settings). I could see this leading to a bit of unnecessary duplication of MIDI events, because in wanting to keep the patterns chaging in tandem with their matching sequence (which might help you keep track of where you are/what you might be editting) you might end up copying much, or all, of the content from another pattern into the new (empty but ‘correctly’ numbered) one. Now, you probably don’t keep auto pattern on and you are thinking that I’m a right patronising git with nothing helpful to say. Or you may be thinking, ‘I like that feature and I don’t make unnecessary duplicates’. And, to be honest, I have wondered whether I ought to mention it. But in the end I found myself wondering about those ‘20 patterns’ and ‘20 sequences’ and couldn’t help thinking maybe you did have auto pattern on and that it might be part of the problem. I think that whilst it can be helpful near the start of your tune-making it may not be later on. The thing is I don’t know why (or even really, if,) you have 20 patterns for each and every track and if the 20 sequences each change a pattern to the corresponding sequence number, or whether it’s just a coincidence that there’s 20 of each. So I probably should just say nothing, but then again, what if me saying something actually helps? I could have just asked but I felt that it might require an answer (and another reply) before we got anywhere. So I decided to risk coming across as a bit patronising. Apologies if I have, I do hope I haven’t.

1 Like

I don’t believe it’s messed up at all. I have not written to Squarp and do not intend to. Pyramid’s support is over: their support time is more valuable on their newer products and I respect that.

I think you nailed it. I have spent a couple of hours doing a lot of testing around on copied versions of this project, and I found the following: if I delete, say, 6 heavily loaded tracks from my project, I can then, from the remaining ones fully copy 1 (heavy) track with its 20 patterns and all its notes again. If I don’t delete the 6 tracks, it won’t let me copy fully that track.
So yes, copy/paste clipboard memory seems impacted by what Pyramid is supposed to cope with elsewhere (loaded tracks existing in project). So there’s a chance data corruption is not on the table (although to be precise, I still need to validate that those 6 tracks are not coincidentally the source of some other bug).

Obviously, i don’t know what your project is like but I can’t help wondering if you really need a new pattern for every track for each sequence change or whether all really need to be 18 bars long.

I’m working on getting Ravel’s Bolero fully electronic (lulz) so yea each bar, pattern, track is different (even velocity-wise).

All your questions were great:

  • yes I have auto-pattern on,
  • but no, I don’t leverage all patterns: I have deleted any pattern where an instrument is not supposed to play. Imagine the flute plays sequences 1-5, then patterns 6 and up are all empty: I am assuming an empty pattern doesn’t consume memory, even if the sequencer switches to its number during play mode.
  • What I did not do is do the extra mile and try to leverage a given pattern several times if it happened to be identical, but I think I might get at best 10% off my project load. You make me realize I could delete some patterns for some instruments (the snare namely) and boil them down to say 5 instead of 20, but I can’t do this for more complex instruments otherwise I would need a written mapping of which patterns should play when (which is super hard to manage when there are 10+ instruments)
  • What I hope, however, is that merging tracks together could ease memory (same amount of midi events, but on fewer tracks).

My plan now is twofold:

  • refine the research and see if deleting patterns (instead of entire tracks) relieves the clipboard too
  • do some cleaning and consolidation, maybe via computer hopefully if MIDI editing is made easier on Reaper (Kenny Gioia to the rescue !!).

I’m happy to keep anyone posted on how Track/Patterns are easy/uneasy to edit via DAW and brought back to the SD Card, if that is of interest.

Thanking this awesome community again.

Two new findings [EDIT]:

  1. Refined research shows it’s indeed not tied to amount of Tracks. If I delete a lot of patterns of a track (except one, so the Track still exists), and I repeat this operation on many tracks, I’m eventually able to copy a heavily loaded track. Memory of the entire project restricts memory of the clipboard.
    1bis) Refined testing shows it’s indeed not tied to amount of Patterns. If I delete all notes within all Patterns (except one, so the Pattern still exists), and I repeat this operation on many Patterns, I’m eventually able to copy a heavily loaded track.
    This means consolidating Patterns between each other on a DAW will not help: the only way out is to reduce the total number of midi events.

  2. An exclamation mark (kind of saying loading, or copying, or pasting, was partial), might overwrite upon save (I’m not sure anymore). Not sure if saving a project despite an exclamation mark might lead to some notes disappearing in the process…

My ask to the Squarp team :pray: (if they ever read this post), is for their future products not to raise warnings in case of memory issues (such as partial load, copy or paste), but to raise a real error. Either the content is copied, loaded, pasted successfully, or it’s not: any in between is unusable for the user imho.

Yes, I think the overall amount of MIDI events in the project is the key. The amount of tracks and patterns in themselves don’t really have any real impact on the memory (it’s what is in them that counts).

You might be able to get away with using some tracks just containing stepped CC automation (parameter locks), say, to control volume of certain channels to enable you to get more mileage out of some of the patterns that contain note information. Maybe you have some patterns that are basically copies of other patterns except they play louder. Instead of having two (or a few) 18 bar patterns you could perhaps get away with one, and a separate automation track with some patterns each with perhaps very few (as little as one?) stepped CC automation changes in them to do the volume (or some other synth parameter) change.

It might be worth seeing what you could do with some of Pyramid’s FX also.

1 Like

@camzeech, I forgot to say that I would be interested to hear about your experiences with importing and exporting MIDI files if you do end up doing that.

And I also meant to ask, if you don’t mind and if you know, what size was your project (in KB). It would be good to have an example of the amount of KB a ‘too big’ project can take. If you could give the total of the project folder and perhaps the how big your core.pyr file is as well. The 13000 MIDI event limit is a bit of a guide but I don’t really know how many MIDI events projects that I’ve made have contained. But I can get to see how many KBs my project folders are on my computer.

Incidentally, I notice that the core.pyr files on the very early test projects I saved were 32KB and that projects a little later on were 64KB (these are from when I backed up my SD card quite a while ago, so my core.pyr files may be quite a bit bigger than that now (which reminds me, I really ought to get and do some more backing up!)). I wonder if that difference in the core.pyr file size has more to do with the fact that I will have saved settings since those early tests or whether it has more to do with having done stuff like enabling patterns, used more tracks, etc.

I did try dropping a few MIDI files into Ableton (Lite) and for each MIDI file it automatically put the patterns on separate MIDI tracks (I don’t really use Ableton other than to record my mixes, so I don’t really know what I’m doing). So I guess the MIDI file contains some information about patterns. It seemed to take into account the length of the patterns and it also asked whether I wanted to import the tempo when I drag and dropped each MIDI file into it, so I guess tempo information is at least included with the MIDI file (though I doubt each track needs that stored individually in Pyramid’s memory). I suppose as each track/pattern can have its own time signature so there may be some (Pyramid) memory used for that per track/pattern (though that may be stored in the ‘core’ rather than the RAM (if those are even different things).

I did have a track that was just one note in one pattern (it was probably used just to trigger a switch in my modular synth). The MIDI file for that was 81 bytes. Another MIDI file with about 35 notes in was 728 bytes. So I suppose that would suggest that tracks/patterns do have a bit of a cost, but that’s a MIDI file and we don’t really know if all of what is in a MIDI file takes up space in the (same) RAM/memory that we can use for MIDI events in our projects. I did also have some MIDI files containing purely CC data. Around 50 changes in CC value took around 600 bytes (in half the number of patterns and a bar less than half the overall length of the MIDI file compared to the 35 note, 728 bytes track).

I suppose Squarp talk about MIDI events in respect of project memory for a reason and looking at the bytes in a project folder might not give that good an impression. Still, I think I would be interested to know what the size of your Bolero project folder was when it approached the limit.

I was sure I read on this forum somewhere that there is some way of seeing how much memory you have remaining on Pyramid (I guess it might be a percentage, which would be sensible and handy). If someone knows if you can, and if so how, it would save me trawling through some old topics on here or through the manual in a more systemmatic way than I already have (I am less sure that I read that now). EDIT: A-ha! It looks like it’s probably in the obvious place, the Info screen (sub-menu?), in the Settings Menu. At least the Quickstart guide in the manual says ‘memory’ is one of the bits of info included there. The Settings Menu section of the manual omits it (though that includes PyrOS version as one bits of info, and the Quickstart doesn’t). I haven’t checked on an actual Pyramid yet though.