Feedback Wanted - Cube Refresh Functionality

by Newcomer Wanting Newcomer 4 weeks ago (685 Views)

We are working on the next generation of MicroStrategy tools that will provide users the ability to manage datasets including refreshing a dataset and schedule a dataset refresh.  To provide a better user experience, we would like to gather feedback from customers who have experience with these functionalities that we currently have on MicroStrategy Web.

 

Please share your use cases by answering the questions below in the comments:

 

Republish Cube

  • How often do you change the refresh policy settings when republishing a cube?
  • How often do you not select all data sources when republishing a cube?
  • Do you expect to refresh multiple cubes at the same time? (i.e. multi-select multiple cubes and refresh)

cuberepub1.png

 

Schedule Cube Refresh

  • How many schedules do you normally have for a cube?
  • If you create more than one schedule for a cube, how often do you do that?
  • What’s the use case of creating multiple schedules on a cube?

cuberepub2.png

 

  • If you have multiple schedules, do you expect to have an interface that allows you to manage all of the schedules, including view details, edit, and delete?
  • As an end user, you can create a schedule for a cube but cannot reuse the schedule for other cubes. To have a reusable schedule template, an Admin has to create one in the metadata. Do you think we should give end users the ability to create schedule templates so that they can be reused later for other cubes?

Comments
by Trusted Expert Trusted Expert
3 weeks ago

The case that I would see is that when you may want incremental refresh on a more frequent basis for performance, but there may be edge cases where the historical data changes, so that you would maybe want a full refresh of the data.

by Senior Guru Senior Guru
3 weeks ago

Presently what you present covert totaly are needs

by BrianFleenor Junior Seeker
3 weeks ago

Republish Cube

  • How often do you change the refresh policy settings when republishing a cube?

Refresh Policy is set once.  Shouldn't change (unless data availability changes and there is a benefit to do so).

  • How often do you not select all data sources when republishing a cube?

Very common.  I have a MTDI cube that I refresh the fact table daily and the dimensional table weekly.  The data's availability is at different intervals.  The fact table has new data available daily while a large dimensional table has new data available weekly.

  • Do you expect to refresh multiple cubes at the same time? (i.e. multi-select multiple cubes and refresh)

My cube refresh process is automated via System Manager using schedules.  I wouldn't expect to manually select multiple cubes and refresh.

 

Schedule Cube Refresh

  • How many schedules do you normally have for a cube?

Two.  With the MTDI cube, one for the daily fact table and a second for the weekly dimensional tables refresh.

  • If you create more than one schedule for a cube, how often do you do that?

Common for MTDI cube.

  • What’s the use case of creating multiple schedules on a cube?

MTDI cubes with varying data availability for each table.

  • If you have multiple schedules, do you expect to have an interface that allows you to manage all of the schedules, including view details, edit, and delete?

It would be helpful to be able to right-click and list all schedules used by the cube and then edit.

  • As an end user, you can create a schedule for a cube but cannot reuse the schedule for other cubes. To have a reusable schedule template, an Admin has to create one in the metadata. Do you think we should give end users the ability to create schedule templates so that they can be reused later for other cubes?

No.  Schedules can get out-of-hand rapidly and creation should remain with administrative oversight.

by Senior Guru Senior Guru
3 weeks ago

We are working for several customer who are replacing / removing cubes in their environments at the moment.

- Cubes are usually loaded when nightly database load has finished. Due to raising data volumes and complexity the ETL time is rising and time for a cube load is missing. End users complain, that their cubes are not ready in the morning. So reports are designed to use data base tables instead of cubes.

- Incremental refresh / update is most often not possible as small changes in facts or dimensions affect the majority rows of a cube. In complex scenarios that are quite common in finance, insurance, re-insurance, a cube usually need to be fully loaded.

- Data volume: A solution for analysis of debts and liabilities usually requires the analysis of an invoice at different time perspectives (key dates). This can hardly be designed as table, data aggregate or cube as the required size would be huge... instead smart views are used. So, several subjects / business units can not use cubes at all...

 

Republish Cube

  • How often do you change the refresh policy settings when republishing a cube?
    • Usually this is defined once by a developer. There is no need to change these settings.
  • How often do you not select all data sources when republishing a cube?
    • Never. Usually all required data source are required for a cube load / cube refresh.
  • Do you expect to refresh multiple cubes at the same time? (i.e. multi-select multiple cubes and refresh)
    • Yes! Almost always! Cubes are triggered when nightly database load for a certain subject has finished.

 

Schedule Cube Refresh

  • How many schedules do you normally have for a cube?
    • One
  • If you create more than one schedule for a cube, how often do you do that?
    • Never. We use triggered schedules
  • What’s the use case of creating multiple schedules on a cube?
    • Different business reasons / technical reasons that require different triggers? We could use one trigger, but maybe monitoring is easier with different trigger and differert schedules...

 

  • If you have multiple schedules, do you expect to have an interface that allows you to manage all of the schedules, including view details, edit, and delete?
    • Yes!
  • As an end user, you can create a schedule for a cube but cannot reuse the schedule for other cubes. To have a reusable schedule template, an Admin has to create one in the metadata. Do you think we should give end users the ability to create schedule templates so that they can be reused later for other cubes?

    • Tricky question! What about providing a privilege that can be assigned to different users, user groups like power users, developers and so on... This privilege should NOT be an administrative privilege. So it can be assigned on project level
by malikmud Pioneer
3 weeks ago

One of the more important features from an end user prespective is our ad-hoc users have the ability to create cubes against their Memory Allocation, We can individually alter a users designated memory limit to set exceptions.

 

We need the ability to unpublish a cube (remove from memory) but keep it available as something that can be republished at a later time if required from a web interface for our self-serve users.

 

Currently the only way to unpublish a cube is to delete the corrosponding files on the server or delete the cube itself.

These also needs to be a way to have cached cubes expire (Unpublish) based on a schedule.

 

by Nachow Pioneer
3 weeks ago

Hi, we have been struggling with the update functionality.

It would be great if you could specify the key by which you will update. Until this exists, we are forced to delete the part of the cube that we want to update and then run the update. This is because any attribute already existing in the cube that changes (for instance any status), actually that row doesn't exist anymore at the datawarehouse but MSTR will insert the new row, leaving the older one as well in the cube.

by Guru Guru
3 weeks ago

Am I the only one with a broken cube importation/refresh functionality in 10.4 ?

It's been broken for months, and support confirmed that currently, users can't import data (except if you give them write permissions on Shared folders Oo)

I was a bit annoyed to see a feedback request on a key feature that's been broken for a very long time, but I see a lot of people answering as if everything was working fine, so I'm wondering : am I the only one affected ?

by Enrique Lopez Junior Contributor
2 weeks ago

 Regarding cube refresh I would like to recommend to enhance MicroStrategy Analytics Enterprise in order to make the Migration more seamless. When i migrated from 10.3 to 10.4, i had to publish all the cubes again, and i find that fact inacceptable for large enterprises, the cube data should be migrated too. 

Also, when refreshing Facebook, twitter, and cloud sources in general after the migration, I found that most of them broke and couldn’t be refreshed anymore. Support kindly asked to re-do all the cubes and dashboards again. That’s inacceptable too, please fix that.

 

Thanks,

by AnthonyC Junior Contributor
2 weeks ago

Republish Cube

  • How often do you change the refresh policy settings when republishing a cube?

If I'm unlucky, I’ll have to do it once a year.

  • How often do you not select all data sources when republishing a cube?

Right now, all of our main cubes are from one data source so by default us admins select all. But there are numerous scenarios where users have created cubes from multiple data sources and they need to update only one of the data sources which is used as a lookup/reference type data source.

  • Do you expect to refresh multiple cubes at the same time? (i.e. multi-select multiple cubes and refresh)

Yes, this is an important in a particular scenario when a database fails. We have to republish a subset of cubes because of failure. We have a command script for all cubes but sometimes we need to update 5-8 at a time. For us we just copy out the command manager lines required, but in a user’s case they may want to refresh a bunch and clicking each one and sending to history is tedious.

 

Schedule Cube Refresh

  • How many schedules do you normally have for a cube?

Usually have one schedule

  • If you create more than one schedule for a cube, how often do you do that?

Once in a while, if database job runs late because of some sort of issue. We will change the schedule of a cube(s) to a later time until the database job performance is fixed.

  • What’s the use case of creating multiple schedules on a cube?

Cube updated frequently during peak work times, less during off peak times for up to date data.

  • If you have multiple schedules, do you expect to have an interface that allows you to manage all of the schedules, including view details, edit, and delete?

Yes, it would be nice if this can be managed in an easy to use interface.

  • As an end user, you can create a schedule for a cube but cannot reuse the schedule for other cubes. To have a reusable schedule template, an Admin has to create one in the metadata. Do you think we should give end users the ability to create schedule templates so that they can be reused later for other cubes?

Right now we don’t have a requirement to do this, but you never know when a request like this will come through from the users.

 

 

by AnthonyC Junior Contributor
2 weeks ago

Am I the only one with a broken cube importation/refresh functionality in 10.4 ?

I have users importing data to cubes in 10.4 HF1, no issues and they don't have write access to the shared folder. They actually do their cube work in their "My Reports" folder. I might be missing something or interpretting your issue incorrectly.  

 

 

 

by Guru Guru
2 weeks ago

@AnthonyC Well we were doing just like you : import in 'my reports' with no write access to shared folders. After migrating to 10.4 we started having an error saying that users don't have permission to create cubes. We contacted support, and they said it was a known bug, going to be fixed in 10.7. Foolishly, I believed them Oo

Can you PM me a screenshot of your users' privileges ? Maybe I missed a new one in 10.4 Oo

by Dirk1520666 Seeker
a week ago

Looking for a faster way to republish cubes at destination after a project copy - something like 'Refresh all cubes' , queue them up and run one after another, or eliminate the need to republish by copying cubes from source (given a full meta-data copy was performed.

by Rogarcia Seeker
Monday

Republish Cube

  • How often do you change the refresh policy settings when republishing a cube?

Never, we use command manager to refresh after data load on the WH.

  • How often do you notselect all data sources when republishing a cube?

We do not need this feature now. All data is from the same data source.

  • Do you expect to refresh multiple cubes at the same time? (i.e. multi-select multiple cubes and refresh)

It would be a great feature. 

Schedule Cube Refresh

  • How many schedules do you normally have for a cube?

None, we refresh it on demand 3 times per day.

  • If you create more than one schedule for a cube, how often do you do that?

Not needed.

  • What’s the use case of creating multiple schedules on a cube?

Dont know

 

  • If you have multiple schedules, do you expect to have an interface that allows you to manage all of the schedules, including view details, edit, and delete?

No

  • As an end user, you can create a schedule for a cube but cannot reuse the schedule for other cubes. To have a reusable schedule template, an Admin has to create one in the metadata. Do you think we should give end users the ability to create schedule templates so that they can be reused later for other cubes?

No

 

What we think it will be a great feature is the possibility to delete and insert records with one refresh command. Now there is no option to delete and then insert, you need to do it in 2 steps
For example if some rows get deleted from the data source table there is no way to delete them from the cube and insert the new data in one step. 
The only way to achieve this is to run 2 incremental refresh reports: one to delete and one to insert the data. Using this method we will have the cube published with no data while the second refresh is made. Example: I have a cube with 3 regions. On the WH region 3 data has many changes, updates, inserts and deletes. So If I want the cube to have the same information as it is on the WH I need to run a refresh cube to delete Region = 3 and then run another refresh cube with insert Region = 3. Between these two refresh cubes there is no data for Region 3.

We did open an ER for this (104182)